2024-09-12T09:15:19.130 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T09:15:19.146 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T09:15:19.238 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510 branch: wip-dis-testing description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: idryomov@gmail.com first_in_suite: false job_id: '7902510' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1680 sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 sleep_before_teardown: 0 subset: 42/100 suite: rbd suite_branch: wip-dis-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIH+lDjbhC6CpJe6Jv2YeNeMAlvtTzr8kVCDO12x4bCRDwCJ7gmlA4E12CjoaMXbw1EKjxDiLY9pPH/N8IL66A= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/ovoZMb77p0OzB6xoyRjkloY1ju7Y7ztFBCU0HB8MXfP3UGSmmXTVjFX1c3sqt2Y0tenxvuqhK42pOQYb7+0c= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-12_04:46:10 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T09:15:19.238 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa; will attempt to use it 2024-09-12T09:15:19.239 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa/tasks 2024-09-12T09:15:19.239 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T09:15:19.241 INFO:teuthology.task.internal:Checking packages... 2024-09-12T09:15:19.262 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1' 2024-09-12T09:15:19.262 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T09:15:19.262 INFO:teuthology.packaging:ref: None 2024-09-12T09:15:19.262 INFO:teuthology.packaging:tag: None 2024-09-12T09:15:19.262 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-12T09:15:19.262 INFO:teuthology.packaging:sha1: d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T09:15:19.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-dis-testing 2024-09-12T09:15:19.442 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:15:19.444 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T09:15:19.463 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T09:15:19.463 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T09:15:19.571 INFO:teuthology.task.internal:Saving configuration 2024-09-12T09:15:19.585 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T09:15:19.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T09:15:19.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 09:09:37.597759', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBj9hJX9LFboof7I5OAGfTisgaqsb7UYC7dFNcSSG6rQ'} 2024-09-12T09:15:19.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 09:09:37.599677', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBT5P341d2dwsI9B3FmlBgVza36gT40Eyag3PDsd5CEe'} 2024-09-12T09:15:19.661 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T09:15:19.678 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-12T09:15:19.678 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-12T09:15:19.678 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T09:15:19.778 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f14e7c17ac0>, signals=[15]) 2024-09-12T09:15:19.778 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T09:15:19.789 INFO:teuthology.task.internal:Opening connections... 2024-09-12T09:15:19.789 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:15:19.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:15:19.884 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:15:19.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:15:19.974 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T09:15:19.983 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-12T09:15:19.988 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-12T09:15:19.989 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-12T09:15:20.043 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-12T09:15:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Ubuntu" 2024-09-12T09:15:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="22.04" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_CODENAME=jammy 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:ID=ubuntu 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE=debian 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-12T09:15:20.132 INFO:teuthology.orchestra.run.smithi007.stdout:UBUNTU_CODENAME=jammy 2024-09-12T09:15:20.133 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-12T09:15:20.165 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-12T09:15:20.171 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-12T09:15:20.171 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Ubuntu" 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="22.04" 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_CODENAME=jammy 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:ID=ubuntu 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE=debian 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-12T09:15:20.217 INFO:teuthology.orchestra.run.smithi115.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-12T09:15:20.218 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-12T09:15:20.218 INFO:teuthology.orchestra.run.smithi115.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-12T09:15:20.218 INFO:teuthology.orchestra.run.smithi115.stdout:UBUNTU_CODENAME=jammy 2024-09-12T09:15:20.218 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-12T09:15:20.257 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T09:15:20.266 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T09:15:20.328 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T09:15:20.328 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-12T09:15:20.331 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-12T09:15:20.337 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T09:15:20.420 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T09:15:20.420 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-12T09:15:20.423 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-12T09:15:20.430 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T09:15:20.519 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T09:15:20.536 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T09:15:20.536 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T09:15:20.537 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T09:15:20.537 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T09:15:20.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:15:20.543 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-12T09:15:20.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:15:20.545 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-12T09:15:20.592 INFO:teuthology.orchestra.run.smithi007.stdout:5.15.0-56-generic 2024-09-12T09:15:20.592 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.15.0-56-generic 2024-09-12T09:15:20.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2024-09-12T09:15:20.594 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-56-generic 2024-09-12T09:15:20.594 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.15.0-56-generic 2024-09-12T09:15:20.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-09-12T09:15:20.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-12T09:15:20.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-12T09:15:20.842 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:15:20.857 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-12T09:15:20.868 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:15:20.886 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-12T09:15:20.925 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-12T09:15:20.949 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-12T09:15:20.957 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T09:15:20.979 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T09:15:21.060 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-12T09:15:21.088 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-12T09:15:21.161 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-12T09:15:21.163 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-12T09:15:21.166 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-12T09:15:21.169 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-12T09:15:21.169 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-12T09:15:21.169 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-12T09:15:21.189 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-12T09:15:21.193 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-12T09:15:21.193 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-12T09:15:21.197 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-12T09:15:21.203 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-12T09:15:21.205 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-12T09:15:21.207 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-12T09:15:21.207 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-12T09:15:21.208 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-12T09:15:21.211 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-12T09:15:21.212 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-12T09:15:21.213 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-12T09:15:21.239 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-12T09:15:21.239 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-12T09:15:21.240 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-12T09:15:21.240 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-12T09:15:21.241 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-12T09:15:21.243 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-12T09:15:21.243 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-12T09:15:21.243 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-12T09:15:21.244 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-12T09:15:21.249 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-12T09:15:21.249 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-12T09:15:21.255 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-12T09:15:21.261 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-12T09:15:21.261 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-12T09:15:21.264 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-12T09:15:21.266 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-12T09:15:21.267 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-12T09:15:21.267 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-12T09:15:21.267 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-12T09:15:21.296 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-12T09:15:21.296 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-12T09:15:21.296 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-12T09:15:21.297 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-12T09:15:21.299 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-12T09:15:21.300 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-12T09:15:21.300 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-12T09:15:21.300 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-12T09:15:21.317 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-12T09:15:21.403 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-12T09:15:21.451 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-12T09:15:21.688 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-12T09:15:21.860 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-12T09:15:21.870 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-12T09:15:21.873 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-12T09:15:21.874 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-12T09:15:21.875 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-12T09:15:21.886 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-12T09:15:21.895 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-12T09:15:21.896 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-12T09:15:21.897 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-12T09:15:21.959 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-12T09:15:21.967 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-12T09:15:21.967 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-12T09:15:21.981 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-12T09:15:21.993 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-12T09:15:22.004 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-12T09:15:22.005 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-12T09:15:22.019 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-12T09:15:22.025 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-12T09:15:22.025 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-12T09:15:22.027 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-12T09:15:22.027 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-12T09:15:22.028 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-12T09:15:22.051 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-12T09:15:22.053 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-12T09:15:22.054 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-12T09:15:22.055 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-12T09:15:22.055 INFO:teuthology.orchestra.run.smithi007.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-12T09:15:22.107 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-12T09:15:22.134 INFO:teuthology.orchestra.run.smithi007.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-12T09:15:27.571 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 15.8 MB in 2s (7,475 kB/s) 2024-09-12T09:15:27.904 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 15.8 MB in 2s (7,307 kB/s) 2024-09-12T09:15:29.061 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:15:29.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T09:15:29.138 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:15:29.339 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-12T09:15:29.339 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-12T09:15:29.348 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:15:29.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T09:15:29.424 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:15:29.507 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:15:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:15:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:15:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:15:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-12T09:15:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: python2.7-minimal 2024-09-12T09:15:29.508 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:15:29.509 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-09-12T09:15:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T09:15:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-12T09:15:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-12T09:15:29.511 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-09-12T09:15:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-12T09:15:29.511 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-09-12T09:15:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: thermald 2024-09-12T09:15:29.557 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-09-12T09:15:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T09:15:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-12T09:15:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-12T09:15:29.559 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages will be upgraded: 2024-09-12T09:15:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-12T09:15:29.611 INFO:teuthology.orchestra.run.smithi115.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T09:15:29.612 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 113 MB of archives. 2024-09-12T09:15:29.612 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-12T09:15:29.612 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-12T09:15:29.619 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-12T09:15:29.620 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-12T09:15:29.792 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:15:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:15:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:15:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:15:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-12T09:15:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: python2.7-minimal 2024-09-12T09:15:29.793 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:15:29.794 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2024-09-12T09:15:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T09:15:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-12T09:15:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-12T09:15:29.795 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2024-09-12T09:15:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-12T09:15:29.795 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2024-09-12T09:15:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: thermald 2024-09-12T09:15:29.841 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2024-09-12T09:15:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T09:15:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-12T09:15:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-12T09:15:29.842 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2024-09-12T09:15:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-12T09:15:29.894 INFO:teuthology.orchestra.run.smithi007.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T09:15:29.894 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 113 MB of archives. 2024-09-12T09:15:29.894 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-12T09:15:29.894 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-12T09:15:30.033 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-12T09:15:30.123 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-12T09:15:30.342 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-12T09:15:30.436 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-12T09:15:30.928 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-12T09:15:30.928 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-12T09:15:30.928 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-12T09:15:31.046 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-12T09:15:31.067 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-12T09:15:31.261 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-12T09:15:31.261 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-12T09:15:31.261 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-12T09:15:31.389 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-12T09:15:31.411 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-12T09:15:31.444 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 113 MB in 2s (75.5 MB/s) 2024-09-12T09:15:31.620 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-12T09:15:31.805 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 113 MB in 2s (72.6 MB/s) 2024-09-12T09:15:31.962 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-12T09:15:33.683 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-12T09:15:33.688 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:33.750 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:33.964 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-12T09:15:33.968 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:34.035 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:35.876 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-12T09:15:35.897 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:35.985 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:36.071 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-12T09:15:36.093 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:36.178 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:36.463 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-12T09:15:36.483 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:36.521 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:36.599 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-12T09:15:36.620 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:36.674 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:43.104 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:43.198 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:43.220 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:43.322 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:43.495 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:43.655 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:43.716 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:43.768 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:44.027 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-12T09:15:44.046 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-12T09:15:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-12T09:15:44.068 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-12T09:15:44.093 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T09:15:44.129 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T09:15:49.410 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-12T09:15:49.448 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:49.485 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:49.515 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-12T09:15:49.552 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T09:15:49.590 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:52.476 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:52.616 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:52.665 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T09:15:52.762 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T09:15:52.950 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T09:15:53.054 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:53.062 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T09:15:53.163 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-12T09:15:53.183 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:53.293 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:53.309 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-12T09:15:53.438 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:15:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-12T09:15:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:15:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-12T09:15:54.797 INFO:teuthology.orchestra.run.smithi007.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:15:54.797 INFO:teuthology.orchestra.run.smithi007.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-12T09:15:54.797 INFO:teuthology.orchestra.run.smithi007.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:15:54.797 INFO:teuthology.orchestra.run.smithi007.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-12T09:15:54.810 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:54.939 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:56.219 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:56.349 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-12T09:15:57.805 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-12T09:15:57.931 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:57.991 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-12T09:15:57.991 INFO:teuthology.orchestra.run.smithi115.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:15:58.626 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-12T09:15:58.752 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-12T09:15:58.878 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T09:15:58.940 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-12T09:15:58.940 INFO:teuthology.orchestra.run.smithi007.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:16:12.867 INFO:teuthology.orchestra.run.smithi115.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-12T09:16:12.867 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub' 2024-09-12T09:16:12.885 INFO:teuthology.orchestra.run.smithi115.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T09:16:12.908 INFO:teuthology.orchestra.run.smithi115.stdout:Generating grub configuration file ... 2024-09-12T09:16:13.161 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:16:13.181 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:16:13.466 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:16:13.472 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T09:16:13.542 INFO:teuthology.orchestra.run.smithi115.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T09:16:13.548 INFO:teuthology.orchestra.run.smithi115.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T09:16:13.719 INFO:teuthology.orchestra.run.smithi115.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T09:16:13.719 INFO:teuthology.orchestra.run.smithi115.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T09:16:13.719 INFO:teuthology.orchestra.run.smithi115.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T09:16:13.756 INFO:teuthology.orchestra.run.smithi115.stdout:done 2024-09-12T09:16:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-12T09:16:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub' 2024-09-12T09:16:13.912 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T09:16:13.943 INFO:teuthology.orchestra.run.smithi007.stdout:Generating grub configuration file ... 2024-09-12T09:16:14.183 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:16:14.192 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-09-12T09:16:14.198 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:16:14.220 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-09-12T09:16:14.220 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 21 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.119.119 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-09-12T09:16:14.221 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-09-12T09:16:14.222 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-09-12T09:16:14.222 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-12T09:16:14.222 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T09:16:14.222 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T09:16:14.222 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T09:16:14.222 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T09:16:14.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get clean 2024-09-12T09:16:14.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-12T09:16:14.495 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:16:14.501 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T09:16:14.529 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:16:14.538 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T09:16:14.540 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T09:16:14.555 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T09:16:14.576 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T09:16:14.582 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T09:16:14.747 INFO:teuthology.orchestra.run.smithi007.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T09:16:14.747 INFO:teuthology.orchestra.run.smithi007.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T09:16:14.747 INFO:teuthology.orchestra.run.smithi007.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T09:16:14.795 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-09-12T09:16:15.252 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2024-09-12T09:16:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2024-09-12T09:16:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2024-09-12T09:16:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2024-09-12T09:16:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 21 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.15.0.119.119 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2024-09-12T09:16:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2024-09-12T09:16:15.277 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-12T09:16:15.277 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T09:16:15.277 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T09:16:15.277 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T09:16:15.277 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T09:16:15.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2024-09-12T09:16:15.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-12T09:16:15.851 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:16:15.851 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T09:16:15.922 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T09:16:15.956 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:16:15.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T09:16:16.003 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T09:16:16.036 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:16:16.258 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-12T09:16:16.258 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-12T09:16:16.437 INFO:teuthology.orchestra.run.smithi115.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-12T09:16:16.437 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:16:16.437 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:16:16.437 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:16:16.437 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:16:16.438 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T09:16:16.438 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T09:16:16.438 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:16:16.480 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T09:16:16.482 DEBUG:teuthology.orchestra.run.smithi115:> dpkg -s linux-image-generic 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Package: linux-image-generic 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Status: install ok installed 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Priority: optional 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Section: kernel 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Installed-Size: 21 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: amd64 2024-09-12T09:16:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:Source: linux-meta 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:Version: 5.15.0.119.119 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:Recommends: thermald 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:Description: Generic Linux kernel image 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout: This package will always depend on the latest generic kernel image 2024-09-12T09:16:16.496 INFO:teuthology.orchestra.run.smithi115.stdout: available. 2024-09-12T09:16:16.496 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-12T09:16:16.543 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.d0Wg0BflnY 2024-09-12T09:16:16.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d0Wg0BflnY 2024-09-12T09:16:16.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.d0Wg0BflnY 2024-09-12T09:16:16.715 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.d0Wg0BflnY is 10KB 2024-09-12T09:16:16.728 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.d0Wg0BflnY 2024-09-12T09:16:16.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-12T09:16:16.786 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:16:16.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-12T09:16:16.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-12T09:16:16.850 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-12T09:16:16.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo update-grub 2024-09-12T09:16:17.314 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:16:17.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T09:16:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:16:17.577 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-12T09:16:17.578 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:16:17.750 INFO:teuthology.orchestra.run.smithi007.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T09:16:17.751 INFO:teuthology.orchestra.run.smithi007.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T09:16:17.751 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:16:17.790 INFO:teuthology.orchestra.run.smithi007.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T09:16:17.793 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2024-09-12T09:16:17.809 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2024-09-12T09:16:17.809 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2024-09-12T09:16:17.809 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2024-09-12T09:16:17.809 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2024-09-12T09:16:17.809 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 21 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.15.0.119.119 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2024-09-12T09:16:17.810 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2024-09-12T09:16:17.810 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-12T09:16:17.855 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.SVFBGRFsKa 2024-09-12T09:16:17.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SVFBGRFsKa 2024-09-12T09:16:17.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.SVFBGRFsKa 2024-09-12T09:16:18.019 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.SVFBGRFsKa is 10KB 2024-09-12T09:16:18.031 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.SVFBGRFsKa 2024-09-12T09:16:18.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-12T09:16:18.083 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub' 2024-09-12T09:16:18.084 INFO:teuthology.orchestra.run.smithi115.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T09:16:18.091 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-12T09:16:18.092 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:16:18.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-12T09:16:18.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-12T09:16:18.181 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-12T09:16:18.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo update-grub 2024-09-12T09:16:18.272 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:16:18.278 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:16:18.515 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:16:18.521 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T09:16:18.563 INFO:teuthology.orchestra.run.smithi115.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T09:16:18.568 INFO:teuthology.orchestra.run.smithi115.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T09:16:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T09:16:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T09:16:18.694 INFO:teuthology.orchestra.run.smithi115.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T09:16:18.715 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-12T09:16:18.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-12T09:16:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub' 2024-09-12T09:16:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T09:16:19.439 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-12T09:16:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T09:16:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T09:16:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T09:16:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T09:16:19.883 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T09:16:19.888 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T09:16:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T09:16:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T09:16:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T09:16:20.012 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-12T09:16:20.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-12T09:16:48.745 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T09:16:48.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:16:48.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:16:50.031 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T09:16:50.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-12T09:16:50.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:17:23.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-12T09:17:32.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:17:32.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:17:50.057 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T09:17:59.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-12T09:17:59.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:18:06.509 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-12T09:18:07.465 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-12T09:18:07.465 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-12T09:18:07.465 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-12T09:18:07.471 INFO:teuthology.orchestra.run.smithi007.stdout:5.15.0-119-generic 2024-09-12T09:18:07.471 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-12T09:18:07.471 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T09:18:07.471 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-12T09:18:08.472 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T09:18:08.472 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T09:18:08.614 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-12T09:18:08.638 DEBUG:teuthology.parallel:result is None 2024-09-12T09:18:32.409 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T09:18:44.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:18:44.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:18:44.725 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-12T09:18:45.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:18:45.346 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-119-generic"... 2024-09-12T09:18:45.347 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-12T09:18:45.392 INFO:teuthology.orchestra.run.smithi115.stdout:5.15.0-119-generic 2024-09-12T09:18:45.393 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-12T09:18:45.393 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T09:18:45.393 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-12T09:18:46.393 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T09:18:46.394 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T09:18:46.490 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-12T09:18:46.514 DEBUG:teuthology.parallel:result is None 2024-09-12T09:18:46.515 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T09:18:46.524 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T09:18:46.524 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T09:18:46.526 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T09:18:46.532 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T09:18:46.565 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T09:18:46.590 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T09:18:46.590 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T09:18:46.593 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T09:18:46.611 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T09:18:46.617 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T09:18:46.617 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T09:18:46.649 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T09:18:46.664 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T09:18:46.667 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T09:18:46.672 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T09:18:46.675 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T09:18:46.677 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T09:18:46.684 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T09:18:46.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T09:18:46.717 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T09:18:46.749 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T09:18:46.759 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T09:18:46.759 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T09:18:46.773 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T09:18:46.797 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T09:18:46.824 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T09:18:46.872 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:18:46.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T09:18:46.954 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T09:18:46.960 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T09:18:47.009 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:18:47.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T09:18:47.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-12T09:18:47.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-12T09:18:47.144 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T09:18:47.152 INFO:teuthology.task.internal:Starting timer... 2024-09-12T09:18:47.152 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T09:18:47.196 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T09:18:47.219 DEBUG:teuthology.task.selinux:Excluding smithi007: OS 'ubuntu' does not support SELinux 2024-09-12T09:18:47.219 DEBUG:teuthology.task.selinux:Excluding smithi115: OS 'ubuntu' does not support SELinux 2024-09-12T09:18:47.220 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T09:18:47.220 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-12T09:18:47.220 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T09:18:47.220 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T09:18:47.251 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T09:18:47.268 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T09:18:47.269 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T09:26:23.406 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2024-09-12T09:26:23.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-12T09:26:23.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:26:23.502 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-12T09:26:23.585 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-12T09:26:23.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:26:23.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T09:26:23.677 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-12T09:26:23.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-12T09:26:23.756 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T09:26:23.771 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T09:26:23.771 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T09:26:23.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T09:26:23.774 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T09:26:23.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T09:26:23.798 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Command line: ntpd -gq 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: ---------------------------------------------------- 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: ntp-4 is maintained by Network Time Foundation, 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: corporation. Support and training for ntp-4 are 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: available at https://www.nwtime.org/support 2024-09-12T09:26:23.799 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: ---------------------------------------------------- 2024-09-12T09:26:23.801 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: proto: precision = 0.064 usec (-24) 2024-09-12T09:26:23.802 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: basedate set to 2022-02-04 2024-09-12T09:26:23.802 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: gps base set to 2022-02-06 (week 2196) 2024-09-12T09:26:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T09:26:23.803 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T09:26:23.803 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: restrict ::: KOD does nothing without LIMITED. 2024-09-12T09:26:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-12T09:26:23.804 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen and drop on 0 v6wildcard [::]:123 2024-09-12T09:26:23.804 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-12T09:26:23.804 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen normally on 2 lo 127.0.0.1:123 2024-09-12T09:26:23.804 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen normally on 3 ens1f0 172.21.15.7:123 2024-09-12T09:26:23.805 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen normally on 4 lo [::1]:123 2024-09-12T09:26:23.805 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15ea%4]:123 2024-09-12T09:26:23.805 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:23 ntpd[18380]: Listening on routing socket on fd #22 for interface updates 2024-09-12T09:26:23.825 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Command line: ntpd -gq 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: ---------------------------------------------------- 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: ntp-4 is maintained by Network Time Foundation, 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: corporation. Support and training for ntp-4 are 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: available at https://www.nwtime.org/support 2024-09-12T09:26:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: ---------------------------------------------------- 2024-09-12T09:26:23.827 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: proto: precision = 0.045 usec (-24) 2024-09-12T09:26:23.827 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: basedate set to 2022-02-04 2024-09-12T09:26:23.827 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: gps base set to 2022-02-06 (week 2196) 2024-09-12T09:26:23.827 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T09:26:23.828 INFO:teuthology.orchestra.run.smithi115.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T09:26:23.828 INFO:teuthology.orchestra.run.smithi115.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-12T09:26:23.829 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: restrict ::: KOD does nothing without LIMITED. 2024-09-12T09:26:23.829 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen and drop on 0 v6wildcard [::]:123 2024-09-12T09:26:23.829 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-12T09:26:23.829 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen normally on 2 lo 127.0.0.1:123 2024-09-12T09:26:23.830 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen normally on 3 enp3s0f1 172.21.15.115:123 2024-09-12T09:26:23.830 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen normally on 4 lo [::1]:123 2024-09-12T09:26:23.830 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce8f%5]:123 2024-09-12T09:26:23.830 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:23 ntpd[18462]: Listening on routing socket on fd #22 for interface updates 2024-09-12T09:26:24.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:24.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:24.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:24.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:24.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:24.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:24 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:25.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:25.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:25 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:26 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:27.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:27 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:27.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:27 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:28.802 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:28.802 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.802 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.802 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:28.802 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:28 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:28.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:28.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:28.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:28.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:28 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:29 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:29.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:29 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stdout:12 Sep 09:26:30 ntpd[18380]: ntpd: time slew -0.003482 s 2024-09-12T09:26:30.800 INFO:teuthology.orchestra.run.smithi007.stdout:ntpd: time slew -0.003482s 2024-09-12T09:26:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-12T09:26:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:12 Sep 09:26:30 ntpd[18380]: can't open /var/log/ntpstats/loopstats.20240912: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:30.826 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:30 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T09:26:31.827 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T09:26:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T09:26:31.828 INFO:teuthology.orchestra.run.smithi115.stdout:12 Sep 09:26:31 ntpd[18462]: ntpd: time slew -0.002442 s 2024-09-12T09:26:31.828 INFO:teuthology.orchestra.run.smithi115.stdout:ntpd: time slew -0.002442s 2024-09-12T09:26:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-12T09:26:31.828 INFO:teuthology.orchestra.run.smithi115.stderr:12 Sep 09:26:31 ntpd[18462]: can't open /var/log/ntpstats/loopstats.20240912: Permission denied 2024-09-12T09:26:31.877 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T09:26:31.877 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-09-12T09:26:31.878 INFO:teuthology.orchestra.run.smithi115.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:31.878 INFO:teuthology.orchestra.run.smithi115.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:31.878 INFO:teuthology.orchestra.run.smithi115.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:31.878 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:26:31.879 INFO:teuthology.run_tasks:Running task install... 2024-09-12T09:26:31.887 DEBUG:teuthology.task.install:project ceph 2024-09-12T09:26:31.887 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}} 2024-09-12T09:26:31.887 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'} 2024-09-12T09:26:31.888 INFO:teuthology.task.install:Using flavor: default 2024-09-12T09:26:31.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T09:26:31.893 INFO:teuthology.task.install:extra packages: [] 2024-09-12T09:26:31.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-key list | grep Ceph 2024-09-12T09:26:31.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-key list | grep Ceph 2024-09-12T09:26:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-12T09:26:31.978 INFO:teuthology.orchestra.run.smithi115.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-12T09:26:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-12T09:26:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph.com (release key) 2024-09-12T09:26:31.994 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-12T09:26:31.995 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T09:26:32.009 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-12T09:26:32.009 INFO:teuthology.orchestra.run.smithi115.stdout:uid [ unknown] Ceph.com (release key) 2024-09-12T09:26:32.009 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-12T09:26:32.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T09:26:32.148 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default/ 2024-09-12T09:26:32.148 INFO:teuthology.task.install.deb:Package version is 19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:26:32.162 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default/ 2024-09-12T09:26:32.162 INFO:teuthology.task.install.deb:Package version is 19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:26:32.271 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:26:32.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-12T09:26:32.278 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:26:32.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-12T09:26:32.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-12T09:26:32.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo apt-get update 2024-09-12T09:26:32.520 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy InRelease 2024-09-12T09:26:32.521 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy InRelease 2024-09-12T09:26:32.542 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-12T09:26:32.542 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-12T09:26:32.565 INFO:teuthology.orchestra.run.smithi115.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-12T09:26:32.566 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-12T09:26:32.590 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:26:32.591 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T09:26:32.591 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-09-12T09:26:32.594 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T09:26:32.594 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T09:26:32.594 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-12T09:26:32.621 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-12T09:26:32.624 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-09-12T09:26:32.660 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T09:26:32.667 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T09:26:32.741 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T09:26:32.752 INFO:teuthology.orchestra.run.smithi115.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T09:26:33.026 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 158 kB in 1s (290 kB/s) 2024-09-12T09:26:33.033 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 158 kB in 1s (288 kB/s) 2024-09-12T09:26:34.060 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:26:34.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4883-gd61b0886-1jammy cephadm=19.3.0-4883-gd61b0886-1jammy ceph-mds=19.3.0-4883-gd61b0886-1jammy ceph-mgr=19.3.0-4883-gd61b0886-1jammy ceph-common=19.3.0-4883-gd61b0886-1jammy ceph-fuse=19.3.0-4883-gd61b0886-1jammy ceph-test=19.3.0-4883-gd61b0886-1jammy ceph-volume=19.3.0-4883-gd61b0886-1jammy radosgw=19.3.0-4883-gd61b0886-1jammy python3-rados=19.3.0-4883-gd61b0886-1jammy python3-rgw=19.3.0-4883-gd61b0886-1jammy python3-cephfs=19.3.0-4883-gd61b0886-1jammy python3-rbd=19.3.0-4883-gd61b0886-1jammy libcephfs2=19.3.0-4883-gd61b0886-1jammy libcephfs-dev=19.3.0-4883-gd61b0886-1jammy librados2=19.3.0-4883-gd61b0886-1jammy librbd1=19.3.0-4883-gd61b0886-1jammy rbd-fuse=19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:26:34.086 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:26:34.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4883-gd61b0886-1jammy cephadm=19.3.0-4883-gd61b0886-1jammy ceph-mds=19.3.0-4883-gd61b0886-1jammy ceph-mgr=19.3.0-4883-gd61b0886-1jammy ceph-common=19.3.0-4883-gd61b0886-1jammy ceph-fuse=19.3.0-4883-gd61b0886-1jammy ceph-test=19.3.0-4883-gd61b0886-1jammy ceph-volume=19.3.0-4883-gd61b0886-1jammy radosgw=19.3.0-4883-gd61b0886-1jammy python3-rados=19.3.0-4883-gd61b0886-1jammy python3-rgw=19.3.0-4883-gd61b0886-1jammy python3-cephfs=19.3.0-4883-gd61b0886-1jammy python3-rbd=19.3.0-4883-gd61b0886-1jammy libcephfs2=19.3.0-4883-gd61b0886-1jammy libcephfs-dev=19.3.0-4883-gd61b0886-1jammy librados2=19.3.0-4883-gd61b0886-1jammy librbd1=19.3.0-4883-gd61b0886-1jammy rbd-fuse=19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:26:34.140 INFO:teuthology.orchestra.run.smithi115.stdout:Reading package lists... 2024-09-12T09:26:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-12T09:26:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-12T09:26:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-12T09:26:34.349 INFO:teuthology.orchestra.run.smithi115.stdout:Building dependency tree... 2024-09-12T09:26:34.350 INFO:teuthology.orchestra.run.smithi115.stdout:Reading state information... 2024-09-12T09:26:34.541 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:26:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2024-09-12T09:26:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-12T09:26:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-12T09:26:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet zip 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-12T09:26:34.546 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2024-09-12T09:26:34.547 INFO:teuthology.orchestra.run.smithi007.stdout: btrfs-tools 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T09:26:34.548 INFO:teuthology.orchestra.run.smithi115.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout:The following additional packages will be installed: 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-12T09:26:34.549 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-12T09:26:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-12T09:26:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-12T09:26:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-12T09:26:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-12T09:26:34.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-12T09:26:34.551 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet zip 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout:Suggested packages: 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-12T09:26:34.552 INFO:teuthology.orchestra.run.smithi115.stdout:Recommended packages: 2024-09-12T09:26:34.553 INFO:teuthology.orchestra.run.smithi115.stdout: btrfs-tools 2024-09-12T09:26:34.651 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2024-09-12T09:26:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-12T09:26:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-12T09:26:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-12T09:26:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-12T09:26:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-12T09:26:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-12T09:26:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-12T09:26:34.656 INFO:teuthology.orchestra.run.smithi115.stdout:The following NEW packages will be installed: 2024-09-12T09:26:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-12T09:26:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-12T09:26:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-12T09:26:34.715 INFO:teuthology.orchestra.run.smithi007.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-12T09:26:34.716 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 251 MB of archives. 2024-09-12T09:26:34.716 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-12T09:26:34.716 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-12T09:26:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-12T09:26:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:Need to get 251 MB of archives. 2024-09-12T09:26:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-12T09:26:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4883-gd61b0886-1jammy [3,634 kB] 2024-09-12T09:26:34.797 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4883-gd61b0886-1jammy [3,634 kB] 2024-09-12T09:26:34.805 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-12T09:26:34.808 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-12T09:26:34.825 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-12T09:26:34.841 INFO:teuthology.orchestra.run.smithi115.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-12T09:26:34.886 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-12T09:26:34.890 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-12T09:26:34.903 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-12T09:26:34.907 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-12T09:26:34.909 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-12T09:26:34.909 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-12T09:26:34.910 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-12T09:26:34.919 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-12T09:26:34.921 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-12T09:26:34.924 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-12T09:26:34.925 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-12T09:26:34.925 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-12T09:26:34.941 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-12T09:26:34.944 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-12T09:26:34.946 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-12T09:26:34.946 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-12T09:26:34.946 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-12T09:26:34.958 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-12T09:26:34.959 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-12T09:26:34.959 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-12T09:26:34.959 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-12T09:26:34.960 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-12T09:26:34.976 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-12T09:26:34.976 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-12T09:26:34.979 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-12T09:26:34.979 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-12T09:26:34.980 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-12T09:26:34.993 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-12T09:26:34.994 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-12T09:26:34.994 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-12T09:26:34.994 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-12T09:26:34.996 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-12T09:26:35.010 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-12T09:26:35.011 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-12T09:26:35.016 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-12T09:26:35.016 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-12T09:26:35.017 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-12T09:26:35.027 INFO:teuthology.orchestra.run.smithi007.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-12T09:26:35.028 INFO:teuthology.orchestra.run.smithi007.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4883-gd61b0886-1jammy [3,336 kB] 2024-09-12T09:26:35.029 INFO:teuthology.orchestra.run.smithi007.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-12T09:26:35.031 INFO:teuthology.orchestra.run.smithi007.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-12T09:26:35.032 INFO:teuthology.orchestra.run.smithi115.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4883-gd61b0886-1jammy [3,336 kB] 2024-09-12T09:26:35.032 INFO:teuthology.orchestra.run.smithi007.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-12T09:26:35.033 INFO:teuthology.orchestra.run.smithi007.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-12T09:26:35.045 INFO:teuthology.orchestra.run.smithi007.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-12T09:26:35.047 INFO:teuthology.orchestra.run.smithi007.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-12T09:26:35.047 INFO:teuthology.orchestra.run.smithi007.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-12T09:26:35.049 INFO:teuthology.orchestra.run.smithi007.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-12T09:26:35.051 INFO:teuthology.orchestra.run.smithi007.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-12T09:26:35.105 INFO:teuthology.orchestra.run.smithi007.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-12T09:26:35.106 INFO:teuthology.orchestra.run.smithi007.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-12T09:26:35.107 INFO:teuthology.orchestra.run.smithi007.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-12T09:26:35.107 INFO:teuthology.orchestra.run.smithi007.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-12T09:26:35.109 INFO:teuthology.orchestra.run.smithi007.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-12T09:26:35.122 INFO:teuthology.orchestra.run.smithi007.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4883-gd61b0886-1jammy [722 kB] 2024-09-12T09:26:35.126 INFO:teuthology.orchestra.run.smithi007.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-12T09:26:35.128 INFO:teuthology.orchestra.run.smithi007.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-12T09:26:35.128 INFO:teuthology.orchestra.run.smithi007.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-12T09:26:35.136 INFO:teuthology.orchestra.run.smithi115.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4883-gd61b0886-1jammy [722 kB] 2024-09-12T09:26:35.143 INFO:teuthology.orchestra.run.smithi007.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-12T09:26:35.144 INFO:teuthology.orchestra.run.smithi007.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-12T09:26:35.144 INFO:teuthology.orchestra.run.smithi007.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4883-gd61b0886-1jammy [356 kB] 2024-09-12T09:26:35.144 INFO:teuthology.orchestra.run.smithi007.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-12T09:26:35.145 INFO:teuthology.orchestra.run.smithi007.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-12T09:26:35.145 INFO:teuthology.orchestra.run.smithi007.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-12T09:26:35.146 INFO:teuthology.orchestra.run.smithi007.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-12T09:26:35.158 INFO:teuthology.orchestra.run.smithi115.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4883-gd61b0886-1jammy [356 kB] 2024-09-12T09:26:35.158 INFO:teuthology.orchestra.run.smithi007.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-12T09:26:35.158 INFO:teuthology.orchestra.run.smithi007.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-12T09:26:35.162 INFO:teuthology.orchestra.run.smithi007.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4883-gd61b0886-1jammy [32.8 kB] 2024-09-12T09:26:35.162 INFO:teuthology.orchestra.run.smithi007.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-12T09:26:35.163 INFO:teuthology.orchestra.run.smithi007.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4883-gd61b0886-1jammy [180 kB] 2024-09-12T09:26:35.167 INFO:teuthology.orchestra.run.smithi007.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4883-gd61b0886-1jammy [71.3 kB] 2024-09-12T09:26:35.168 INFO:teuthology.orchestra.run.smithi007.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4883-gd61b0886-1jammy [340 kB] 2024-09-12T09:26:35.176 INFO:teuthology.orchestra.run.smithi007.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-12T09:26:35.176 INFO:teuthology.orchestra.run.smithi007.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4883-gd61b0886-1jammy [7,987 kB] 2024-09-12T09:26:35.176 INFO:teuthology.orchestra.run.smithi115.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4883-gd61b0886-1jammy [32.8 kB] 2024-09-12T09:26:35.177 INFO:teuthology.orchestra.run.smithi007.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-12T09:26:35.177 INFO:teuthology.orchestra.run.smithi115.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4883-gd61b0886-1jammy [180 kB] 2024-09-12T09:26:35.177 INFO:teuthology.orchestra.run.smithi007.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-12T09:26:35.180 INFO:teuthology.orchestra.run.smithi007.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-12T09:26:35.181 INFO:teuthology.orchestra.run.smithi115.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4883-gd61b0886-1jammy [71.3 kB] 2024-09-12T09:26:35.182 INFO:teuthology.orchestra.run.smithi007.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-12T09:26:35.183 INFO:teuthology.orchestra.run.smithi115.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4883-gd61b0886-1jammy [340 kB] 2024-09-12T09:26:35.183 INFO:teuthology.orchestra.run.smithi007.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-12T09:26:35.184 INFO:teuthology.orchestra.run.smithi007.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-12T09:26:35.190 INFO:teuthology.orchestra.run.smithi115.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4883-gd61b0886-1jammy [7,987 kB] 2024-09-12T09:26:35.193 INFO:teuthology.orchestra.run.smithi007.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-12T09:26:35.193 INFO:teuthology.orchestra.run.smithi007.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-12T09:26:35.196 INFO:teuthology.orchestra.run.smithi007.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-12T09:26:35.210 INFO:teuthology.orchestra.run.smithi007.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-12T09:26:35.210 INFO:teuthology.orchestra.run.smithi007.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-12T09:26:35.210 INFO:teuthology.orchestra.run.smithi007.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-12T09:26:35.212 INFO:teuthology.orchestra.run.smithi007.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-12T09:26:35.213 INFO:teuthology.orchestra.run.smithi007.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-12T09:26:35.219 INFO:teuthology.orchestra.run.smithi007.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-12T09:26:35.219 INFO:teuthology.orchestra.run.smithi007.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-12T09:26:35.227 INFO:teuthology.orchestra.run.smithi007.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-12T09:26:35.229 INFO:teuthology.orchestra.run.smithi007.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-12T09:26:35.229 INFO:teuthology.orchestra.run.smithi007.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-12T09:26:35.260 INFO:teuthology.orchestra.run.smithi115.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-12T09:26:35.272 INFO:teuthology.orchestra.run.smithi115.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-12T09:26:35.353 INFO:teuthology.orchestra.run.smithi115.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-12T09:26:35.438 INFO:teuthology.orchestra.run.smithi007.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4883-gd61b0886-1jammy [112 kB] 2024-09-12T09:26:35.441 INFO:teuthology.orchestra.run.smithi007.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4883-gd61b0886-1jammy [473 kB] 2024-09-12T09:26:35.450 INFO:teuthology.orchestra.run.smithi007.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4883-gd61b0886-1jammy [28.3 MB] 2024-09-12T09:26:35.462 INFO:teuthology.orchestra.run.smithi115.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4883-gd61b0886-1jammy [112 kB] 2024-09-12T09:26:35.465 INFO:teuthology.orchestra.run.smithi115.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4883-gd61b0886-1jammy [473 kB] 2024-09-12T09:26:35.485 INFO:teuthology.orchestra.run.smithi115.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4883-gd61b0886-1jammy [28.3 MB] 2024-09-12T09:26:35.581 INFO:teuthology.orchestra.run.smithi115.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-12T09:26:35.591 INFO:teuthology.orchestra.run.smithi115.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-12T09:26:35.621 INFO:teuthology.orchestra.run.smithi115.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-12T09:26:35.630 INFO:teuthology.orchestra.run.smithi115.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-12T09:26:35.632 INFO:teuthology.orchestra.run.smithi115.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-12T09:26:35.633 INFO:teuthology.orchestra.run.smithi115.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-12T09:26:35.634 INFO:teuthology.orchestra.run.smithi115.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-12T09:26:35.652 INFO:teuthology.orchestra.run.smithi115.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-12T09:26:35.656 INFO:teuthology.orchestra.run.smithi115.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-12T09:26:35.663 INFO:teuthology.orchestra.run.smithi115.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-12T09:26:35.742 INFO:teuthology.orchestra.run.smithi115.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-12T09:26:35.742 INFO:teuthology.orchestra.run.smithi115.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-12T09:26:35.749 INFO:teuthology.orchestra.run.smithi115.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-12T09:26:35.751 INFO:teuthology.orchestra.run.smithi115.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-12T09:26:35.754 INFO:teuthology.orchestra.run.smithi115.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-12T09:26:35.755 INFO:teuthology.orchestra.run.smithi115.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-12T09:26:35.756 INFO:teuthology.orchestra.run.smithi115.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-12T09:26:35.758 INFO:teuthology.orchestra.run.smithi115.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-12T09:26:35.759 INFO:teuthology.orchestra.run.smithi115.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-12T09:26:35.826 INFO:teuthology.orchestra.run.smithi115.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-12T09:26:35.841 INFO:teuthology.orchestra.run.smithi115.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-12T09:26:35.842 INFO:teuthology.orchestra.run.smithi115.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-12T09:26:35.842 INFO:teuthology.orchestra.run.smithi115.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-12T09:26:35.843 INFO:teuthology.orchestra.run.smithi115.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-12T09:26:35.848 INFO:teuthology.orchestra.run.smithi115.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-12T09:26:35.849 INFO:teuthology.orchestra.run.smithi115.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-12T09:26:35.907 INFO:teuthology.orchestra.run.smithi115.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-12T09:26:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-12T09:26:35.910 INFO:teuthology.orchestra.run.smithi115.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-12T09:26:35.988 INFO:teuthology.orchestra.run.smithi115.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-12T09:26:35.988 INFO:teuthology.orchestra.run.smithi115.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-12T09:26:35.990 INFO:teuthology.orchestra.run.smithi115.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-12T09:26:35.992 INFO:teuthology.orchestra.run.smithi115.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-12T09:26:35.993 INFO:teuthology.orchestra.run.smithi115.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-12T09:26:36.004 INFO:teuthology.orchestra.run.smithi115.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-12T09:26:36.069 INFO:teuthology.orchestra.run.smithi115.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-12T09:26:36.070 INFO:teuthology.orchestra.run.smithi115.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-12T09:26:36.071 INFO:teuthology.orchestra.run.smithi115.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-12T09:26:36.073 INFO:teuthology.orchestra.run.smithi115.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-12T09:26:36.150 INFO:teuthology.orchestra.run.smithi115.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-12T09:26:36.151 INFO:teuthology.orchestra.run.smithi115.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-12T09:26:36.154 INFO:teuthology.orchestra.run.smithi115.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-12T09:26:36.155 INFO:teuthology.orchestra.run.smithi115.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-12T09:26:36.161 INFO:teuthology.orchestra.run.smithi115.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-12T09:26:36.161 INFO:teuthology.orchestra.run.smithi115.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-12T09:26:36.231 INFO:teuthology.orchestra.run.smithi115.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-12T09:26:36.236 INFO:teuthology.orchestra.run.smithi115.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-12T09:26:36.398 INFO:teuthology.orchestra.run.smithi115.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-12T09:26:36.400 INFO:teuthology.orchestra.run.smithi115.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-12T09:26:36.402 INFO:teuthology.orchestra.run.smithi115.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-12T09:26:36.403 INFO:teuthology.orchestra.run.smithi115.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-12T09:26:36.405 INFO:teuthology.orchestra.run.smithi115.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-12T09:26:36.514 INFO:teuthology.orchestra.run.smithi115.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-12T09:26:36.568 INFO:teuthology.orchestra.run.smithi007.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4883-gd61b0886-1jammy [5,207 kB] 2024-09-12T09:26:36.600 INFO:teuthology.orchestra.run.smithi115.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-12T09:26:36.600 INFO:teuthology.orchestra.run.smithi115.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4883-gd61b0886-1jammy [5,207 kB] 2024-09-12T09:26:36.600 INFO:teuthology.orchestra.run.smithi115.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-12T09:26:36.631 INFO:teuthology.orchestra.run.smithi115.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-12T09:26:36.631 INFO:teuthology.orchestra.run.smithi115.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-12T09:26:36.633 INFO:teuthology.orchestra.run.smithi115.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-12T09:26:36.634 INFO:teuthology.orchestra.run.smithi115.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-12T09:26:36.634 INFO:teuthology.orchestra.run.smithi115.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-12T09:26:36.635 INFO:teuthology.orchestra.run.smithi115.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-12T09:26:36.645 INFO:teuthology.orchestra.run.smithi115.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-12T09:26:36.650 INFO:teuthology.orchestra.run.smithi007.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4883-gd61b0886-1jammy [251 kB] 2024-09-12T09:26:36.653 INFO:teuthology.orchestra.run.smithi007.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4883-gd61b0886-1jammy [124 kB] 2024-09-12T09:26:36.654 INFO:teuthology.orchestra.run.smithi007.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4883-gd61b0886-1jammy [1,562 kB] 2024-09-12T09:26:36.668 INFO:teuthology.orchestra.run.smithi115.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4883-gd61b0886-1jammy [251 kB] 2024-09-12T09:26:36.670 INFO:teuthology.orchestra.run.smithi007.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4883-gd61b0886-1jammy [6,309 kB] 2024-09-12T09:26:36.674 INFO:teuthology.orchestra.run.smithi115.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4883-gd61b0886-1jammy [124 kB] 2024-09-12T09:26:36.675 INFO:teuthology.orchestra.run.smithi115.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4883-gd61b0886-1jammy [1,562 kB] 2024-09-12T09:26:36.718 INFO:teuthology.orchestra.run.smithi115.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-12T09:26:36.726 INFO:teuthology.orchestra.run.smithi115.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-12T09:26:36.728 INFO:teuthology.orchestra.run.smithi115.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4883-gd61b0886-1jammy [6,309 kB] 2024-09-12T09:26:36.730 INFO:teuthology.orchestra.run.smithi115.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-12T09:26:36.734 INFO:teuthology.orchestra.run.smithi115.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-12T09:26:36.735 INFO:teuthology.orchestra.run.smithi115.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-12T09:26:36.743 INFO:teuthology.orchestra.run.smithi115.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-12T09:26:36.750 INFO:teuthology.orchestra.run.smithi115.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-12T09:26:36.752 INFO:teuthology.orchestra.run.smithi115.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-12T09:26:36.754 INFO:teuthology.orchestra.run.smithi115.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-12T09:26:36.808 INFO:teuthology.orchestra.run.smithi115.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-12T09:26:36.846 INFO:teuthology.orchestra.run.smithi115.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-12T09:26:36.853 INFO:teuthology.orchestra.run.smithi115.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-12T09:26:36.854 INFO:teuthology.orchestra.run.smithi115.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-12T09:26:36.856 INFO:teuthology.orchestra.run.smithi115.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-12T09:26:36.856 INFO:teuthology.orchestra.run.smithi115.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-12T09:26:36.884 INFO:teuthology.orchestra.run.smithi007.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4883-gd61b0886-1jammy [23.1 MB] 2024-09-12T09:26:36.890 INFO:teuthology.orchestra.run.smithi115.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-12T09:26:36.892 INFO:teuthology.orchestra.run.smithi115.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-12T09:26:36.909 INFO:teuthology.orchestra.run.smithi115.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-12T09:26:36.909 INFO:teuthology.orchestra.run.smithi115.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-12T09:26:36.937 INFO:teuthology.orchestra.run.smithi115.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4883-gd61b0886-1jammy [23.1 MB] 2024-09-12T09:26:36.970 INFO:teuthology.orchestra.run.smithi115.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-12T09:26:36.975 INFO:teuthology.orchestra.run.smithi115.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-12T09:26:36.976 INFO:teuthology.orchestra.run.smithi115.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-12T09:26:37.737 INFO:teuthology.orchestra.run.smithi007.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4883-gd61b0886-1jammy [14.1 kB] 2024-09-12T09:26:37.737 INFO:teuthology.orchestra.run.smithi007.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4883-gd61b0886-1jammy [880 kB] 2024-09-12T09:26:37.754 INFO:teuthology.orchestra.run.smithi007.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4883-gd61b0886-1jammy [2,454 kB] 2024-09-12T09:26:37.793 INFO:teuthology.orchestra.run.smithi115.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4883-gd61b0886-1jammy [14.1 kB] 2024-09-12T09:26:37.794 INFO:teuthology.orchestra.run.smithi115.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4883-gd61b0886-1jammy [880 kB] 2024-09-12T09:26:37.797 INFO:teuthology.orchestra.run.smithi007.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4883-gd61b0886-1jammy [814 kB] 2024-09-12T09:26:37.808 INFO:teuthology.orchestra.run.smithi007.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4883-gd61b0886-1jammy [160 kB] 2024-09-12T09:26:37.810 INFO:teuthology.orchestra.run.smithi007.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4883-gd61b0886-1jammy [61.5 MB] 2024-09-12T09:26:37.811 INFO:teuthology.orchestra.run.smithi115.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4883-gd61b0886-1jammy [2,454 kB] 2024-09-12T09:26:37.852 INFO:teuthology.orchestra.run.smithi115.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4883-gd61b0886-1jammy [814 kB] 2024-09-12T09:26:37.862 INFO:teuthology.orchestra.run.smithi115.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4883-gd61b0886-1jammy [160 kB] 2024-09-12T09:26:37.864 INFO:teuthology.orchestra.run.smithi115.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4883-gd61b0886-1jammy [61.5 MB] 2024-09-12T09:26:40.105 INFO:teuthology.orchestra.run.smithi007.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4883-gd61b0886-1jammy [8,625 kB] 2024-09-12T09:26:40.173 INFO:teuthology.orchestra.run.smithi115.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4883-gd61b0886-1jammy [8,625 kB] 2024-09-12T09:26:40.219 INFO:teuthology.orchestra.run.smithi007.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4883-gd61b0886-1jammy [14.2 kB] 2024-09-12T09:26:40.219 INFO:teuthology.orchestra.run.smithi007.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4883-gd61b0886-1jammy [55.8 MB] 2024-09-12T09:26:40.362 INFO:teuthology.orchestra.run.smithi115.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4883-gd61b0886-1jammy [14.2 kB] 2024-09-12T09:26:40.362 INFO:teuthology.orchestra.run.smithi115.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4883-gd61b0886-1jammy [55.8 MB] 2024-09-12T09:26:42.259 INFO:teuthology.orchestra.run.smithi007.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4883-gd61b0886-1jammy [130 kB] 2024-09-12T09:26:42.261 INFO:teuthology.orchestra.run.smithi007.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4883-gd61b0886-1jammy [39.0 kB] 2024-09-12T09:26:42.325 INFO:teuthology.orchestra.run.smithi007.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4883-gd61b0886-1jammy [15.9 MB] 2024-09-12T09:26:42.438 INFO:teuthology.orchestra.run.smithi115.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4883-gd61b0886-1jammy [130 kB] 2024-09-12T09:26:42.440 INFO:teuthology.orchestra.run.smithi115.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4883-gd61b0886-1jammy [39.0 kB] 2024-09-12T09:26:42.441 INFO:teuthology.orchestra.run.smithi115.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4883-gd61b0886-1jammy [15.9 MB] 2024-09-12T09:26:42.677 INFO:teuthology.orchestra.run.smithi007.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4883-gd61b0886-1jammy [91.2 kB] 2024-09-12T09:26:42.827 INFO:teuthology.orchestra.run.smithi115.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/d61b08865d4d98738f45ff2e4806d7fb6d03eca1/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4883-gd61b0886-1jammy [91.2 kB] 2024-09-12T09:26:43.287 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 251 MB in 8s (31.3 MB/s) 2024-09-12T09:26:43.399 INFO:teuthology.orchestra.run.smithi115.stdout:Fetched 251 MB in 8s (30.8 MB/s) 2024-09-12T09:26:43.482 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-12T09:26:43.529 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-12T09:26:43.579 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-12T09:26:43.584 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-12T09:26:43.627 INFO:teuthology.orchestra.run.smithi115.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-12T09:26:43.632 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-12T09:26:43.672 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T09:26:43.702 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T09:26:44.024 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-12T09:26:44.026 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-12T09:26:44.050 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-12T09:26:44.065 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-12T09:26:44.090 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T09:26:44.094 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T09:26:44.379 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-12T09:26:44.406 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-12T09:26:44.417 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-12T09:26:44.437 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T09:26:44.443 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-12T09:26:44.484 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T09:26:44.891 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-12T09:26:44.895 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-12T09:26:44.918 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:44.922 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:44.963 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:44.965 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:45.349 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-12T09:26:45.375 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:45.377 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-12T09:26:45.404 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:45.423 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:45.444 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:45.760 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-12T09:26:45.786 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:45.822 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-12T09:26:45.826 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:45.849 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T09:26:45.897 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:26:46.121 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-12T09:26:46.144 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-12T09:26:46.180 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T09:26:46.292 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-12T09:26:46.319 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-12T09:26:46.367 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T09:26:46.491 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librados2. 2024-09-12T09:26:46.517 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../007-librados2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:46.557 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librados2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:46.678 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librados2. 2024-09-12T09:26:46.705 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../007-librados2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:46.745 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librados2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:47.035 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libnbd0. 2024-09-12T09:26:47.062 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-12T09:26:47.102 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-12T09:26:47.396 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librbd1. 2024-09-12T09:26:47.423 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../009-librbd1_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:47.463 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librbd1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:47.565 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libnbd0. 2024-09-12T09:26:47.591 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-12T09:26:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-12T09:26:47.874 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs2. 2024-09-12T09:26:47.901 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:47.941 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.009 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librbd1. 2024-09-12T09:26:48.036 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../009-librbd1_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:48.084 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librbd1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.243 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rados. 2024-09-12T09:26:48.258 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:48.301 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rados (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.546 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs2. 2024-09-12T09:26:48.572 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:48.604 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-12T09:26:48.612 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.630 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:26:48.671 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-argparse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.898 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rados. 2024-09-12T09:26:48.899 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cephfs. 2024-09-12T09:26:48.914 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:48.925 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:48.948 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cephfs (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:48.965 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rados (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:49.261 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-12T09:26:49.288 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:26:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-12T09:26:49.320 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:26:49.328 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:49.359 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-argparse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:49.603 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cephfs. 2024-09-12T09:26:49.619 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:49.647 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-12T09:26:49.653 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cephfs (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:49.674 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-12T09:26:49.714 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T09:26:49.940 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-12T09:26:49.966 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:26:50.006 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:50.025 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-prettytable. 2024-09-12T09:26:50.052 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-12T09:26:50.092 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-12T09:26:50.328 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rbd. 2024-09-12T09:26:50.354 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:50.368 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-12T09:26:50.386 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rbd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-12T09:26:50.443 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T09:26:50.713 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-12T09:26:50.740 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-12T09:26:50.780 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T09:26:50.804 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-prettytable. 2024-09-12T09:26:50.831 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-12T09:26:50.879 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-12T09:26:51.157 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rbd. 2024-09-12T09:26:51.184 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:51.223 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rbd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:51.449 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-12T09:26:51.476 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-12T09:26:51.516 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T09:26:51.645 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-12T09:26:51.672 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-12T09:26:51.720 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T09:26:51.810 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-12T09:26:51.825 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-12T09:26:51.869 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T09:26:52.047 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-12T09:26:52.074 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-12T09:26:52.139 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T09:26:52.163 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua5.1. 2024-09-12T09:26:52.190 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-12T09:26:52.221 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-12T09:26:52.459 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-12T09:26:52.475 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-12T09:26:52.526 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T09:26:52.533 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-any. 2024-09-12T09:26:52.559 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-12T09:26:52.599 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-12T09:26:52.852 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package zip. 2024-09-12T09:26:52.878 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-12T09:26:52.912 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package lua5.1. 2024-09-12T09:26:52.918 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking zip (3.0-12build2) ... 2024-09-12T09:26:52.939 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-12T09:26:52.987 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-12T09:26:53.246 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package unzip. 2024-09-12T09:26:53.273 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-12T09:26:53.313 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-12T09:26:53.315 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package lua-any. 2024-09-12T09:26:53.342 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-12T09:26:53.382 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-12T09:26:53.609 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package zip. 2024-09-12T09:26:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-12T09:26:53.650 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package luarocks. 2024-09-12T09:26:53.676 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking zip (3.0-12build2) ... 2024-09-12T09:26:53.676 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-12T09:26:53.708 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-12T09:26:54.021 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package unzip. 2024-09-12T09:26:54.047 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-12T09:26:54.065 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package librgw2. 2024-09-12T09:26:54.092 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../026-librgw2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:54.095 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-12T09:26:54.132 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking librgw2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:54.541 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package luarocks. 2024-09-12T09:26:54.568 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-12T09:26:54.607 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-12T09:26:54.751 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rgw. 2024-09-12T09:26:54.778 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:54.817 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:55.007 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librgw2. 2024-09-12T09:26:55.034 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../026-librgw2_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:55.065 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librgw2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:55.154 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-12T09:26:55.180 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-12T09:26:55.220 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T09:26:55.473 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libradosstriper1. 2024-09-12T09:26:55.488 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:55.523 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libradosstriper1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rgw. 2024-09-12T09:26:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:56.091 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-common. 2024-09-12T09:26:56.095 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-12T09:26:56.117 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:56.122 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-12T09:26:56.162 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T09:26:56.276 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:56.448 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libradosstriper1. 2024-09-12T09:26:56.475 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:56.514 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libradosstriper1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:56.909 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-common. 2024-09-12T09:26:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:56.976 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:57.789 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-base. 2024-09-12T09:26:57.816 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:57.866 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-base (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:58.460 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-12T09:26:58.486 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-12T09:26:58.514 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-base. 2024-09-12T09:26:58.526 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-12T09:26:58.541 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:26:58.590 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-base (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:26:58.829 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cheroot. 2024-09-12T09:26:58.856 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-12T09:26:58.895 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T09:26:59.143 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-12T09:26:59.170 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-12T09:26:59.210 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-12T09:26:59.223 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-12T09:26:59.250 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-12T09:26:59.290 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-12T09:26:59.496 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cheroot. 2024-09-12T09:26:59.523 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-12T09:26:59.551 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-12T09:26:59.563 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T09:26:59.567 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-12T09:26:59.601 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-12T09:26:59.904 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-12T09:26:59.916 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-12T09:26:59.930 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-12T09:26:59.943 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-12T09:26:59.970 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-12T09:26:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-12T09:27:00.287 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-12T09:27:00.290 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempora. 2024-09-12T09:27:00.312 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-12T09:27:00.316 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-12T09:27:00.352 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-12T09:27:00.356 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-12T09:27:00.630 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-12T09:27:00.650 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-portend. 2024-09-12T09:27:00.656 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-12T09:27:00.677 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-12T09:27:00.696 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-12T09:27:00.725 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-12T09:27:00.961 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-12T09:27:00.965 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-tempora. 2024-09-12T09:27:00.988 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-12T09:27:00.992 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-12T09:27:01.028 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-12T09:27:01.032 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-12T09:27:01.335 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-portend. 2024-09-12T09:27:01.362 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-12T09:27:01.401 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-12T09:27:01.471 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-12T09:27:01.498 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-12T09:27:01.581 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-12T09:27:01.704 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-12T09:27:01.731 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-12T09:27:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-12T09:27:01.901 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-natsort. 2024-09-12T09:27:01.927 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-12T09:27:01.959 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-12T09:27:02.032 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-12T09:27:02.059 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-12T09:27:02.091 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-12T09:27:02.245 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-logutils. 2024-09-12T09:27:02.272 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-12T09:27:02.312 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-12T09:27:02.452 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-natsort. 2024-09-12T09:27:02.479 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-12T09:27:02.518 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-12T09:27:02.623 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-12T09:27:02.650 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-12T09:27:02.689 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-12T09:27:02.830 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-logutils. 2024-09-12T09:27:02.857 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-12T09:27:02.897 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-12T09:27:03.001 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-mako. 2024-09-12T09:27:03.027 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-12T09:27:03.067 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T09:27:03.183 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-12T09:27:03.210 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-12T09:27:03.249 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-12T09:27:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-12T09:27:03.413 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-12T09:27:03.453 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-12T09:27:03.519 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-mako. 2024-09-12T09:27:03.546 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-12T09:27:03.577 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T09:27:03.698 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-12T09:27:03.713 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-12T09:27:03.747 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-12T09:27:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-12T09:27:03.899 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-12T09:27:03.947 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-12T09:27:04.042 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webob. 2024-09-12T09:27:04.068 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-12T09:27:04.108 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T09:27:04.249 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-12T09:27:04.265 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-12T09:27:04.308 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-12T09:27:04.461 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-waitress. 2024-09-12T09:27:04.488 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-12T09:27:04.531 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T09:27:04.603 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webob. 2024-09-12T09:27:04.630 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-12T09:27:04.669 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T09:27:04.858 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-tempita. 2024-09-12T09:27:04.883 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-12T09:27:04.931 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T09:27:05.072 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-waitress. 2024-09-12T09:27:05.099 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-12T09:27:05.150 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T09:27:05.209 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-paste. 2024-09-12T09:27:05.235 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-12T09:27:05.275 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T09:27:05.466 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-tempita. 2024-09-12T09:27:05.493 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-12T09:27:05.533 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T09:27:05.603 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-12T09:27:05.630 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-12T09:27:05.670 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T09:27:05.836 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-paste. 2024-09-12T09:27:05.863 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-12T09:27:05.902 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T09:27:05.972 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-12T09:27:05.999 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-12T09:27:06.039 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-12T09:27:06.205 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-12T09:27:06.221 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-12T09:27:06.264 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T09:27:06.308 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-webtest. 2024-09-12T09:27:06.335 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-12T09:27:06.366 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-12T09:27:06.566 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-12T09:27:06.594 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-12T09:27:06.633 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-12T09:27:06.653 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pecan. 2024-09-12T09:27:06.679 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-12T09:27:06.719 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T09:27:07.040 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-certifi. 2024-09-12T09:27:07.067 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-12T09:27:07.107 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-12T09:27:07.427 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-idna. 2024-09-12T09:27:07.453 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-12T09:27:07.493 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T09:27:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webtest. 2024-09-12T09:27:08.053 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-12T09:27:08.095 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-12T09:27:08.373 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pecan. 2024-09-12T09:27:08.400 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-12T09:27:08.431 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T09:27:08.580 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-urllib3. 2024-09-12T09:27:08.607 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-12T09:27:08.671 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T09:27:08.761 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-certifi. 2024-09-12T09:27:08.788 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-12T09:27:08.819 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-12T09:27:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests. 2024-09-12T09:27:08.992 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-12T09:27:09.032 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T09:27:09.130 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-idna. 2024-09-12T09:27:09.157 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-12T09:27:09.197 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T09:27:09.293 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-12T09:27:09.320 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-12T09:27:09.351 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T09:27:09.458 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-urllib3. 2024-09-12T09:27:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-12T09:27:09.516 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T09:27:09.646 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-12T09:27:09.661 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:09.704 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:09.802 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-requests. 2024-09-12T09:27:09.818 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-12T09:27:09.852 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T09:27:10.092 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-12T09:27:10.119 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:10.159 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:10.163 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-12T09:27:10.190 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-12T09:27:10.230 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T09:27:10.470 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-12T09:27:10.496 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-12T09:27:10.524 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-12T09:27:10.536 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-12T09:27:10.540 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:10.583 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:10.806 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-openssl. 2024-09-12T09:27:10.832 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-12T09:27:10.872 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-12T09:27:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-12T09:27:11.006 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:11.046 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:11.150 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr. 2024-09-12T09:27:11.176 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:11.208 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:11.365 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-12T09:27:11.392 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-12T09:27:11.432 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-12T09:27:11.619 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mon. 2024-09-12T09:27:11.646 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:11.686 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mon (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:11.735 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-openssl. 2024-09-12T09:27:11.762 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-12T09:27:11.801 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-12T09:27:12.129 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr. 2024-09-12T09:27:12.157 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:12.187 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:12.281 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-osd. 2024-09-12T09:27:12.308 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:12.347 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-osd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:12.582 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mon. 2024-09-12T09:27:12.609 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:12.648 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mon (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:13.261 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-osd. 2024-09-12T09:27:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:13.327 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-osd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:13.392 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph. 2024-09-12T09:27:13.407 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../067-ceph_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:13.450 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:13.753 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-fuse. 2024-09-12T09:27:13.780 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:13.819 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:14.181 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mds. 2024-09-12T09:27:14.207 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:14.247 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mds (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:14.422 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph. 2024-09-12T09:27:14.438 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../067-ceph_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:14.472 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:14.675 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package cephadm. 2024-09-12T09:27:14.701 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../070-cephadm_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:14.741 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:14.749 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-fuse. 2024-09-12T09:27:14.776 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:14.816 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:15.002 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-12T09:27:15.018 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-12T09:27:15.052 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-12T09:27:15.202 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mds. 2024-09-12T09:27:15.229 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:15.268 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mds (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:15.405 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-12T09:27:15.420 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T09:27:15.471 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:15.672 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephadm. 2024-09-12T09:27:15.699 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../070-cephadm_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:15.738 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:16.016 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-12T09:27:16.032 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-12T09:27:16.066 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-12T09:27:16.402 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-12T09:27:16.418 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T09:27:16.460 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:16.476 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-babel. 2024-09-12T09:27:16.504 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T09:27:16.543 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:16.862 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-jinja2. 2024-09-12T09:27:16.889 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-12T09:27:17.068 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T09:27:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-babel. 2024-09-12T09:27:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T09:27:17.498 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-12T09:27:17.515 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:17.525 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:17.564 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:17.859 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jinja2. 2024-09-12T09:27:17.887 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-12T09:27:17.901 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-12T09:27:17.917 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T09:27:17.928 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-12T09:27:17.984 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-12T09:27:18.229 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-12T09:27:18.245 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:18.279 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:18.296 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-routes. 2024-09-12T09:27:18.322 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-12T09:27:18.362 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T09:27:18.565 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-12T09:27:18.581 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-12T09:27:18.615 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-12T09:27:18.698 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-12T09:27:18.725 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:18.765 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:18.893 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-routes. 2024-09-12T09:27:18.920 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-12T09:27:18.959 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T09:27:19.279 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-12T09:27:19.307 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:21.536 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-12T09:27:21.553 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-12T09:27:21.594 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T09:27:22.056 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-joblib. 2024-09-12T09:27:22.083 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-12T09:27:22.122 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T09:27:22.234 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-12T09:27:22.258 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-12T09:27:22.300 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T09:27:22.484 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-12T09:27:22.511 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-12T09:27:22.550 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-12T09:27:22.787 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-joblib. 2024-09-12T09:27:22.815 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-12T09:27:22.845 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T09:27:22.853 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-sklearn. 2024-09-12T09:27:22.880 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-12T09:27:23.005 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T09:27:23.266 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-12T09:27:23.283 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-12T09:27:23.341 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-12T09:27:23.689 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-12T09:27:23.717 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:23.894 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn. 2024-09-12T09:27:23.923 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-12T09:27:24.080 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T09:27:24.716 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-cachetools. 2024-09-12T09:27:24.744 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-12T09:27:24.791 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-12T09:27:25.103 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-12T09:27:25.130 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-12T09:27:25.148 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-12T09:27:25.169 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-12T09:27:25.177 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:25.214 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:25.498 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-12T09:27:25.525 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-12T09:27:25.564 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T09:27:25.874 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-rsa. 2024-09-12T09:27:25.891 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-12T09:27:25.941 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-12T09:27:26.176 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cachetools. 2024-09-12T09:27:26.204 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-12T09:27:26.220 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-google-auth. 2024-09-12T09:27:26.242 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-12T09:27:26.247 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-12T09:27:26.278 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-12T09:27:26.581 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-12T09:27:26.609 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-12T09:27:26.612 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-12T09:27:26.640 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-12T09:27:26.647 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T09:27:26.686 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-12T09:27:26.959 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-websocket. 2024-09-12T09:27:26.987 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-12T09:27:27.025 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-12T09:27:27.082 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-12T09:27:27.110 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-12T09:27:27.156 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T09:27:27.337 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-12T09:27:27.364 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-12T09:27:27.403 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T09:27:27.534 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rsa. 2024-09-12T09:27:27.563 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-12T09:27:27.601 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-12T09:27:27.921 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-google-auth. 2024-09-12T09:27:27.950 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-12T09:27:28.071 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-12T09:27:28.391 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-12T09:27:28.420 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-12T09:27:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T09:27:28.527 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-12T09:27:28.556 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:28.594 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:28.752 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-websocket. 2024-09-12T09:27:28.769 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-12T09:27:28.802 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-12T09:27:28.913 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-12T09:27:28.941 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-12T09:27:28.979 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T09:27:29.180 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-12T09:27:29.209 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-12T09:27:29.247 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T09:27:29.316 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-12T09:27:29.344 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T09:27:29.382 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T09:27:29.618 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package jq. 2024-09-12T09:27:29.646 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T09:27:29.693 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-12T09:27:29.894 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-12T09:27:29.911 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:29.929 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package socat. 2024-09-12T09:27:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-12T09:27:29.952 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:29.979 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-12T09:27:30.265 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package xmlstarlet. 2024-09-12T09:27:30.280 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-12T09:27:30.282 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-12T09:27:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-12T09:27:30.324 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-12T09:27:30.346 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T09:27:30.662 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-test. 2024-09-12T09:27:30.689 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:30.728 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-test (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:30.733 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-12T09:27:30.762 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T09:27:30.807 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T09:27:31.110 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package jq. 2024-09-12T09:27:31.139 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T09:27:31.185 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-12T09:27:31.457 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package socat. 2024-09-12T09:27:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-12T09:27:31.523 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-12T09:27:31.868 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package xmlstarlet. 2024-09-12T09:27:31.885 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-12T09:27:31.935 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-12T09:27:32.314 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-test. 2024-09-12T09:27:32.331 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:32.372 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-test (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:33.224 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package ceph-volume. 2024-09-12T09:27:33.251 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:33.290 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking ceph-volume (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:33.611 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-12T09:27:33.639 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:33.677 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking libcephfs-dev (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:33.938 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-12T09:27:33.954 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-12T09:27:33.988 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T09:27:34.309 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-12T09:27:34.337 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-12T09:27:34.376 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-12T09:27:34.747 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package nvme-cli. 2024-09-12T09:27:34.775 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-12T09:27:34.813 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T09:27:34.944 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-volume. 2024-09-12T09:27:34.972 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4883-gd61b0886-1jammy_all.deb ... 2024-09-12T09:27:35.010 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-volume (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:35.218 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package pkg-config. 2024-09-12T09:27:35.246 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-12T09:27:35.285 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T09:27:35.314 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-12T09:27:35.331 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:35.372 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs-dev (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:35.563 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-12T09:27:35.579 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-12T09:27:35.621 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-12T09:27:35.658 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-12T09:27:35.687 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-12T09:27:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T09:27:36.013 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-12T09:27:36.030 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-12T09:27:36.032 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-12T09:27:36.060 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-12T09:27:36.062 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-12T09:27:36.098 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-12T09:27:36.401 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pastescript. 2024-09-12T09:27:36.429 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-12T09:27:36.434 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package nvme-cli. 2024-09-12T09:27:36.462 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-12T09:27:36.476 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-12T09:27:36.500 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T09:27:36.805 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pluggy. 2024-09-12T09:27:36.833 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-12T09:27:36.871 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-12T09:27:36.939 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package pkg-config. 2024-09-12T09:27:36.968 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-12T09:27:37.006 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T09:27:37.149 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-psutil. 2024-09-12T09:27:37.173 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-12T09:27:37.207 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-12T09:27:37.267 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-12T09:27:37.284 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-12T09:27:37.325 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-12T09:27:37.552 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-py. 2024-09-12T09:27:37.580 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-12T09:27:37.618 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-12T09:27:37.762 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-12T09:27:37.790 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-12T09:27:37.828 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-12T09:27:37.963 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pygments. 2024-09-12T09:27:37.991 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-12T09:27:38.030 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T09:27:38.098 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastescript. 2024-09-12T09:27:38.127 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-12T09:27:38.156 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-12T09:27:38.484 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-12T09:27:38.484 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pluggy. 2024-09-12T09:27:38.512 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-12T09:27:38.513 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-12T09:27:38.550 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-12T09:27:38.551 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-12T09:27:38.836 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-toml. 2024-09-12T09:27:38.865 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-12T09:27:38.903 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-12T09:27:39.087 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-psutil. 2024-09-12T09:27:39.104 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-12T09:27:39.137 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-12T09:27:39.206 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-pytest. 2024-09-12T09:27:39.234 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-12T09:27:39.272 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T09:27:39.507 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-py. 2024-09-12T09:27:39.536 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-12T09:27:39.581 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-12T09:27:39.626 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package python3-simplejson. 2024-09-12T09:27:39.654 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-12T09:27:39.692 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-12T09:27:39.877 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pygments. 2024-09-12T09:27:39.894 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-12T09:27:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T09:27:39.961 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-12T09:27:39.977 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-12T09:27:40.019 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-12T09:27:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-12T09:27:40.410 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-12T09:27:40.456 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-12T09:27:40.682 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package radosgw. 2024-09-12T09:27:40.710 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../117-radosgw_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:40.734 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-toml. 2024-09-12T09:27:40.748 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking radosgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:40.762 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-12T09:27:40.808 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-12T09:27:41.112 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pytest. 2024-09-12T09:27:41.140 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-12T09:27:41.178 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T09:27:41.457 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplejson. 2024-09-12T09:27:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-12T09:27:41.515 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-12T09:27:41.651 INFO:teuthology.orchestra.run.smithi115.stdout:Selecting previously unselected package rbd-fuse. 2024-09-12T09:27:41.668 INFO:teuthology.orchestra.run.smithi115.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:41.709 INFO:teuthology.orchestra.run.smithi115.stdout:Unpacking rbd-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:41.759 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-12T09:27:41.776 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-12T09:27:41.817 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-12T09:27:42.006 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-12T09:27:42.239 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T09:27:42.365 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T09:27:42.522 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package radosgw. 2024-09-12T09:27:42.551 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../117-radosgw_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:42.588 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking radosgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:42.625 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-12T09:27:42.948 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-12T09:27:43.381 INFO:teuthology.orchestra.run.smithi115.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-12T09:27:43.525 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package rbd-fuse. 2024-09-12T09:27:43.542 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4883-gd61b0886-1jammy_amd64.deb ... 2024-09-12T09:27:43.583 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking rbd-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:43.633 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:43.888 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-12T09:27:44.138 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T09:27:44.192 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user cephadm....done 2024-09-12T09:27:44.256 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T09:27:44.348 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T09:27:44.524 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-12T09:27:44.649 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-12T09:27:44.849 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-12T09:27:44.917 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-12T09:27:45.043 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-12T09:27:45.273 INFO:teuthology.orchestra.run.smithi007.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-12T09:27:45.294 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-12T09:27:45.349 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:45.562 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T09:27:45.663 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-12T09:27:45.841 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user cephadm....done 2024-09-12T09:27:45.981 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T09:27:45.997 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T09:27:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-12T09:27:46.375 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-12T09:27:46.542 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-12T09:27:46.643 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-12T09:27:46.668 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-12T09:27:46.911 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-12T09:27:46.919 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-12T09:27:47.154 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-12T09:27:47.162 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T09:27:47.263 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-12T09:27:47.388 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-12T09:27:47.573 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T09:27:47.656 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-12T09:27:47.942 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-12T09:27:48.040 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-argparse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:48.210 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-12T09:27:48.308 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T09:27:48.459 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-12T09:27:48.486 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-12T09:27:48.594 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T09:27:48.713 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-12T09:27:48.719 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T09:27:48.863 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-12T09:27:48.997 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-12T09:27:49.130 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-12T09:27:49.274 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-12T09:27:49.512 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-12T09:27:49.533 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-argparse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:49.545 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-12T09:27:49.608 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T09:27:49.743 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-12T09:27:49.943 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T09:27:50.052 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-12T09:27:50.094 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T09:27:50.195 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-12T09:27:50.262 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T09:27:50.372 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-12T09:27:50.446 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T09:27:50.640 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-12T09:27:50.722 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-12T09:27:51.010 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-12T09:27:51.046 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-12T09:27:51.048 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up zip (3.0-12build2) ... 2024-09-12T09:27:51.110 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T09:27:51.199 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T09:27:51.211 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-12T09:27:51.554 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T09:27:51.679 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-12T09:27:51.885 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T09:27:51.948 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T09:27:52.136 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T09:27:52.215 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-12T09:27:52.278 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-12T09:27:52.429 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T09:27:52.516 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up zip (3.0-12build2) ... 2024-09-12T09:27:52.643 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T09:27:52.738 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-12T09:27:53.123 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-12T09:27:53.320 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T09:27:53.383 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T09:27:53.604 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T09:27:53.730 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-12T09:27:53.759 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T09:27:53.865 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T09:27:54.052 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T09:27:54.182 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-12T09:27:54.379 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-12T09:27:54.609 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-12T09:27:54.646 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-12T09:27:54.772 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:54.893 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T09:27:55.082 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T09:27:55.270 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T09:27:55.450 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T09:27:55.563 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T09:27:55.914 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-12T09:27:56.165 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-12T09:27:56.291 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:27:56.601 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T09:27:56.631 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T09:27:56.907 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:27:56.910 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T09:27:57.075 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-12T09:27:57.343 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T09:27:57.444 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-12T09:27:57.545 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T09:27:57.838 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-12T09:27:58.092 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T09:27:58.156 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-12T09:27:58.335 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:27:58.424 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:27:58.444 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-12T09:27:58.575 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-12T09:27:58.704 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T09:27:58.830 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-12T09:27:58.964 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T09:27:58.976 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-12T09:27:59.219 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-12T09:27:59.266 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-12T09:27:59.487 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-12T09:27:59.525 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-12T09:27:59.755 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-12T09:27:59.759 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:27:59.861 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-12T09:28:00.030 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-12T09:28:00.162 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-12T09:28:00.298 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T09:28:00.422 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-12T09:28:00.449 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-12T09:28:00.690 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-12T09:28:00.742 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T09:28:00.860 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T09:28:00.949 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-12T09:28:01.145 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T09:28:01.209 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-12T09:28:01.485 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T09:28:01.496 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-12T09:28:01.586 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-12T09:28:01.771 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T09:28:01.887 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T09:28:01.898 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-12T09:28:02.013 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T09:28:02.023 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T09:28:02.113 INFO:teuthology.orchestra.run.smithi115.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-12T09:28:02.306 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T09:28:02.351 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T09:28:02.477 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T09:28:02.632 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-12T09:28:02.891 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T09:28:02.895 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-12T09:28:03.001 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-12T09:28:03.102 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T09:28:03.162 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-12T09:28:03.174 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-12T09:28:03.314 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-12T09:28:03.396 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T09:28:03.521 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T09:28:03.590 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:28:03.716 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T09:28:03.915 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-12T09:28:04.034 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T09:28:04.182 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-12T09:28:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-12T09:28:04.326 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T09:28:04.551 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T09:28:04.611 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-12T09:28:04.686 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T09:28:04.720 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-12T09:28:04.987 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T09:28:04.988 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-12T09:28:05.288 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T09:28:05.375 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T09:28:05.598 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-12T09:28:05.634 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-12T09:28:05.724 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-12T09:28:06.000 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-12T09:28:06.011 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-12T09:28:06.304 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T09:28:06.353 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T09:28:06.621 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-12T09:28:06.656 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T09:28:06.790 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librados2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:06.908 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.009 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-12T09:28:07.023 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-12T09:28:07.312 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.319 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T09:28:07.438 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libradosstriper1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.564 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-12T09:28:07.686 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T09:28:07.812 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librados2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.840 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librbd1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.938 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:07.975 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-modules-core (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.072 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-12T09:28:08.076 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.274 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T09:28:08.274 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T09:28:08.409 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.518 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libradosstriper1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.619 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-12T09:28:08.769 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up libcephfs-dev (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.895 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rados (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:08.904 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librbd1 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:09.030 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-modules-core (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:09.063 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up librgw2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:09.156 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:09.190 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T09:28:09.377 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T09:28:09.378 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T09:28:10.039 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs-dev (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:10.167 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rados (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:10.318 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librgw2 (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:10.419 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T09:28:10.694 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rbd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:10.811 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up rbd-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:10.937 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-rgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:11.063 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up python3-cephfs (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:11.189 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:11.682 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rbd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:11.699 INFO:teuthology.orchestra.run.smithi115.stdout:Adding group ceph....done 2024-09-12T09:28:11.808 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up rbd-fuse (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:11.917 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:12.018 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cephfs (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:12.034 INFO:teuthology.orchestra.run.smithi115.stdout:Adding system user ceph....done 2024-09-12T09:28:12.119 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-common (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:12.211 INFO:teuthology.orchestra.run.smithi115.stdout:Setting system user ceph properties....done 2024-09-12T09:28:12.224 INFO:teuthology.orchestra.run.smithi115.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-12T09:28:12.361 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-12T09:28:12.662 INFO:teuthology.orchestra.run.smithi007.stdout:Adding group ceph....done 2024-09-12T09:28:12.720 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-12T09:28:13.055 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user ceph....done 2024-09-12T09:28:13.174 INFO:teuthology.orchestra.run.smithi007.stdout:Setting system user ceph properties....done 2024-09-12T09:28:13.183 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-test (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:13.186 INFO:teuthology.orchestra.run.smithi007.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-12T09:28:13.284 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up radosgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:13.341 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-12T09:28:13.694 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-12T09:28:13.716 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T09:28:13.716 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T09:28:14.152 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-base (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:14.171 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-test (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:14.305 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up radosgw (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:14.799 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T09:28:14.799 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T09:28:14.815 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-12T09:28:15.280 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mds (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-base (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:15.509 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T09:28:15.509 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T09:28:15.646 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-12T09:28:15.982 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:16.151 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mds (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:16.241 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T09:28:16.242 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T09:28:16.375 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T09:28:16.375 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T09:28:16.753 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-osd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:17.074 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T09:28:17.074 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T09:28:17.131 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T09:28:17.131 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T09:28:17.570 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:17.662 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-osd (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:17.696 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:17.847 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mon (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:17.942 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T09:28:17.942 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T09:28:18.075 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T09:28:18.075 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T09:28:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.523 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.556 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.649 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.708 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mon (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.750 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-mgr-dashboard (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.885 INFO:teuthology.orchestra.run.smithi115.stdout:Setting up ceph-volume (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:18.917 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T09:28:18.917 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T09:28:19.230 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T09:28:19.330 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T09:28:19.384 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-cephadm (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:19.482 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T09:28:19.518 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:19.644 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-dashboard (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:19.795 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-volume (19.3.0-4883-gd61b0886-1jammy) ... 2024-09-12T09:28:20.166 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T09:28:20.275 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T09:28:20.283 INFO:teuthology.orchestra.run.smithi115.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T09:28:20.419 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T09:28:20.936 INFO:teuthology.orchestra.run.smithi115.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T09:28:20.942 DEBUG:teuthology.parallel:result is None 2024-09-12T09:28:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T09:28:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T09:28:22.192 DEBUG:teuthology.parallel:result is None 2024-09-12T09:28:22.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T09:28:22.355 DEBUG:teuthology.orchestra.run.smithi007:> dpkg-query -W -f '${Version}' ceph 2024-09-12T09:28:22.375 INFO:teuthology.orchestra.run.smithi007.stdout:19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:28:22.376 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:28:22.376 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883-gd61b0886-1jammy is installed. 2024-09-12T09:28:22.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d61b08865d4d98738f45ff2e4806d7fb6d03eca1 2024-09-12T09:28:22.530 DEBUG:teuthology.orchestra.run.smithi115:> dpkg-query -W -f '${Version}' ceph 2024-09-12T09:28:22.549 INFO:teuthology.orchestra.run.smithi115.stdout:19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:28:22.550 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883-gd61b0886-1jammy 2024-09-12T09:28:22.550 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883-gd61b0886-1jammy is installed. 2024-09-12T09:28:22.552 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T09:28:22.552 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:22.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T09:28:22.567 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:22.567 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T09:28:22.604 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T09:28:22.605 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:22.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T09:28:22.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T09:28:22.680 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:22.680 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T09:28:22.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T09:28:22.747 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T09:28:22.747 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:22.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T09:28:22.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T09:28:22.816 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:22.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T09:28:22.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T09:28:22.883 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T09:28:22.883 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:22.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T09:28:22.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T09:28:22.952 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:22.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T09:28:22.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T09:28:23.018 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T09:28:23.105 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T09:28:23.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-12T09:28:23.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-09-12T09:28:23.120 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T09:28:23.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T09:28:23.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T09:28:23.175 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T09:28:23.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T09:28:23.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T09:28:23.242 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-12T09:28:23.242 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T09:28:23.242 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510', 'branch': 'wip-dis-testing', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7902510', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1680, 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'sleep_before_teardown': 0, 'subset': '42/100', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d61b08865d4d98738f45ff2e4806d7fb6d03eca1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIH+lDjbhC6CpJe6Jv2YeNeMAlvtTzr8kVCDO12x4bCRDwCJ7gmlA4E12CjoaMXbw1EKjxDiLY9pPH/N8IL66A=', 'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/ovoZMb77p0OzB6xoyRjkloY1ju7Y7ztFBCU0HB8MXfP3UGSmmXTVjFX1c3sqt2Y0tenxvuqhK42pOQYb7+0c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-12_04:46:10', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T09:28:23.243 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T09:28:23.273 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T09:28:23.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T09:28:23.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T09:28:23.337 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:23.337 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T09:28:23.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T09:28:23.380 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-12 09:22:48.502202480 +0000 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-12 09:22:48.226204947 +0000 2024-09-12T09:28:23.429 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-12 09:22:48.226204947 +0000 2024-09-12T09:28:23.430 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-12T09:28:23.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T09:28:23.483 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-12T09:28:23.483 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-12T09:28:23.483 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000311258 s, 1.6 MB/s 2024-09-12T09:28:23.485 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T09:28:23.534 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-12T09:28:23.580 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-12 09:22:48.994198083 +0000 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-12 09:22:48.730200441 +0000 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-12 09:22:48.730200441 +0000 2024-09-12T09:28:23.581 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-12T09:28:23.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T09:28:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-12T09:28:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-12T09:28:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00032961 s, 1.6 MB/s 2024-09-12T09:28:23.636 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T09:28:23.686 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-12T09:28:23.732 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 912 Links: 1 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-12 09:22:49.558193041 +0000 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-12 09:22:49.310195259 +0000 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-12 09:22:49.310195259 +0000 2024-09-12T09:28:23.733 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-12T09:28:23.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T09:28:23.786 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-12T09:28:23.786 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-12T09:28:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000273292 s, 1.9 MB/s 2024-09-12T09:28:23.788 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T09:28:23.838 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-12T09:28:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-12 09:22:49.802190861 +0000 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-12 09:22:49.798190897 +0000 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-12 09:22:49.798190897 +0000 2024-09-12T09:28:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-12T09:28:23.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T09:28:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-12T09:28:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-12T09:28:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000198738 s, 2.6 MB/s 2024-09-12T09:28:23.941 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T09:28:23.990 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:23.990 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:23.990 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T09:28:23.995 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T09:28:23.996 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 09:22:47.811881021 +0000 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 09:22:47.559883263 +0000 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 09:22:47.559883263 +0000 2024-09-12T09:28:24.045 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-12T09:28:24.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T09:28:24.098 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T09:28:24.099 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T09:28:24.099 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000458394 s, 1.1 MB/s 2024-09-12T09:28:24.100 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T09:28:24.153 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 09:22:48.303876641 +0000 2024-09-12T09:28:24.200 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 09:22:48.043878955 +0000 2024-09-12T09:28:24.201 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 09:22:48.043878955 +0000 2024-09-12T09:28:24.201 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-12T09:28:24.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T09:28:24.254 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T09:28:24.254 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T09:28:24.254 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000416867 s, 1.2 MB/s 2024-09-12T09:28:24.255 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T09:28:24.305 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-12T09:28:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T09:28:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:24.352 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-12T09:28:24.352 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:24.352 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 09:22:48.803872191 +0000 2024-09-12T09:28:24.353 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 09:22:48.531874613 +0000 2024-09-12T09:28:24.353 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 09:22:48.531874613 +0000 2024-09-12T09:28:24.353 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-12T09:28:24.353 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T09:28:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T09:28:24.406 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T09:28:24.407 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000441921 s, 1.2 MB/s 2024-09-12T09:28:24.408 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T09:28:24.457 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 941 Links: 1 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-12 09:22:49.271868026 +0000 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-12 09:22:49.023870233 +0000 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-12 09:22:49.023870233 +0000 2024-09-12T09:28:24.504 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2024-09-12T09:28:24.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T09:28:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-12T09:28:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-12T09:28:24.558 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000385902 s, 1.3 MB/s 2024-09-12T09:28:24.560 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T09:28:24.609 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:24.609 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T09:28:24.609 INFO:tasks.ceph:Generating config... 2024-09-12T09:28:24.611 INFO:tasks.ceph:[client] rbd cache = True 2024-09-12T09:28:24.611 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-09-12T09:28:24.611 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-12T09:28:24.611 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-12T09:28:24.611 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-12T09:28:24.611 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T09:28:24.611 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T09:28:24.611 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T09:28:24.611 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T09:28:24.611 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-12T09:28:24.612 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T09:28:24.613 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-12T09:28:24.613 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T09:28:24.613 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-12T09:28:24.613 INFO:tasks.ceph:Setting up mon.a... 2024-09-12T09:28:24.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-12T09:28:24.649 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-12T09:28:24.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-12T09:28:24.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T09:28:24.796 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.c', '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]'), ('mon.b', '172.21.15.115')] 2024-09-12T09:28:24.796 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.115', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-12T09:28:24.797 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:24.797 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T09:28:24.844 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --addv c '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]' --add b 172.21.15.115 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:24.932 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 177484da-1bae-43aa-8977-8d19d52d569f 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 177484da-1bae-43aa-8977-8d19d52d569f 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-12T09:28:24.933015+0000 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-12T09:28:24.933015+0000 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-12T09:28:24.933 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-12T09:28:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-09-12T09:28:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] mon.c 2024-09-12T09:28:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-12T09:28:24.935 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T09:28:24.940 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 177484da-1bae-43aa-8977-8d19d52d569f... 2024-09-12T09:28:24.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T09:28:24.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.016 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T09:28:25.017 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-12T09:28:25.018 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.115" 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 5000 2024-09-12T09:28:25.019 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 177484da-1bae-43aa-8977-8d19d52d569f 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-12T09:28:25.020 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.021 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression algorithm = zstd 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression mode = aggressive 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-12T09:28:25.022 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: osd shutdown pgref assert = True 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-12T09:28:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-12T09:28:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.025 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: rbd cache = True 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout: rbd cache policy = writearound 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-12T09:28:25.026 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.028 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T09:28:25.029 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-09-12T09:28:25.030 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.115" 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on pool no app = False 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 5000 2024-09-12T09:28:25.031 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 177484da-1bae-43aa-8977-8d19d52d569f 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-09-12T09:28:25.032 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug trim objects = true 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-09-12T09:28:25.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression algorithm = zstd 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression mode = aggressive 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 1/20 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 1/20 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 4/10 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T09:28:25.034 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: osd blocked scrub grace period = 3600 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: osd max markdown count = 1000 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: osd shutdown pgref assert = True 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-12T09:28:25.035 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-09-12T09:28:25.036 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-09-12T09:28:25.037 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: rbd cache = True 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout: rbd cache policy = writearound 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-09-12T09:28:25.038 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-09-12T09:28:25.038 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-12T09:28:25.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-12T09:28:25.122 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T09:28:25.122 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:25.122 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-12T09:28:25.127 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:25.127 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-12T09:28:25.176 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:28:25.176 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:25.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T09:28:25.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T09:28:25.242 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:25.242 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:25.289 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:28:25.289 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:25.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T09:28:25.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T09:28:25.311 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:25.312 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:25.360 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T09:28:25.360 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T09:28:25.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T09:28:25.406 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T09:28:25.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T09:28:25.452 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T09:28:25.466 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T09:28:25.466 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T09:28:25.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T09:28:25.502 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-12T09:28:25.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-12T09:28:25.546 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-12T09:28:25.556 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T09:28:25.556 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T09:28:25.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-12T09:28:25.569 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:25.569 INFO:tasks.ceph:role: osd.0 2024-09-12T09:28:25.569 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:28:25.569 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T09:28:25.624 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:25.648 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-12T09:28:25.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-12T09:28:25.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-12T09:28:25.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-12T09:28:25.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:25.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-12T09:28:25.813 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:25.814 INFO:tasks.ceph:role: osd.1 2024-09-12T09:28:25.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:28:25.814 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T09:28:25.868 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:25.869 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:25.892 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-12T09:28:25.894 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-12T09:28:25.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-12T09:28:25.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-12T09:28:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:26.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:26.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-12T09:28:26.064 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:26.065 INFO:tasks.ceph:role: osd.2 2024-09-12T09:28:26.065 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:28:26.065 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:26.121 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:26.122 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:26.145 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-12T09:28:26.147 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-12T09:28:26.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-12T09:28:26.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-12T09:28:26.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:26.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-12T09:28:26.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:26.311 INFO:tasks.ceph:role: osd.3 2024-09-12T09:28:26.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:28:26.312 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:26.368 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:26.369 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:26.369 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:26.392 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-12T09:28:26.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-12T09:28:26.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-12T09:28:26.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-12T09:28:26.506 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:26.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:26.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:26.626+0000 7f05823e07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-12T09:28:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:26.626+0000 7f05823e07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-12T09:28:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:26.626+0000 7f05823e07c0 -1 bdev(0x558992901800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:26.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:26.626+0000 7f05823e07c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-12T09:28:29.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-12T09:28:29.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:29.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:29.862+0000 7f5c5bdee7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-12T09:28:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:29.866+0000 7f5c5bdee7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-12T09:28:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:29.866+0000 7f5c5bdee7c0 -1 bdev(0x558d49559800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:29.866+0000 7f5c5bdee7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-12T09:28:33.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-12T09:28:33.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:33.162+0000 7f0f350b37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-12T09:28:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:33.162+0000 7f0f350b37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-12T09:28:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:33.162+0000 7f0f350b37c0 -1 bdev(0x556301269800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:33.162+0000 7f0f350b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-12T09:28:36.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-12T09:28:36.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:36.426+0000 7f18ba7077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-12T09:28:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:36.426+0000 7f18ba7077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-12T09:28:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:36.426+0000 7f18ba7077c0 -1 bdev(0x563be18c1800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:28:36.426+0000 7f18ba7077c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-12T09:28:39.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-12T09:28:39.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-12T09:28:39.635 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:39.635 INFO:tasks.ceph:role: osd.4 2024-09-12T09:28:39.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:28:39.636 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T09:28:39.700 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:39.700 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:39.701 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:39.701 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:39.701 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:40.154 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-12T09:28:40.156 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-12T09:28:40.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-12T09:28:40.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-12T09:28:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:40.259 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:40.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-12T09:28:40.315 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:40.315 INFO:tasks.ceph:role: osd.5 2024-09-12T09:28:40.315 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:28:40.315 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:40.382 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:40.861 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-12T09:28:40.864 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-12T09:28:40.864 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-12T09:28:41.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-12T09:28:41.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:41.024 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:41.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-12T09:28:41.083 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:41.083 INFO:tasks.ceph:role: osd.6 2024-09-12T09:28:41.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:28:41.083 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T09:28:41.147 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:41.147 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:41.148 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:41.632 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-12T09:28:41.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-12T09:28:41.634 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-12T09:28:41.789 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-12T09:28:41.801 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:41.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:41.802 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-12T09:28:41.855 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T09:28:41.855 INFO:tasks.ceph:role: osd.7 2024-09-12T09:28:41.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:28:41.856 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T09:28:41.919 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T09:28:41.919 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T09:28:41.920 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T09:28:42.349 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-12T09:28:42.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-12T09:28:42.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-12T09:28:42.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-12T09:28:42.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:28:42.453 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T09:28:42.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:42.558 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:42.554+0000 7f18c09e77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-12T09:28:42.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:42.558+0000 7f18c09e77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-12T09:28:42.559 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:42.558+0000 7f18c09e77c0 -1 bdev(0x55c87bf15800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:42.560 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:42.558+0000 7f18c09e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-12T09:28:45.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-12T09:28:45.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:45.784 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:45.782+0000 7f33f48717c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-12T09:28:45.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:45.782+0000 7f33f48717c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-12T09:28:45.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:45.782+0000 7f33f48717c0 -1 bdev(0x5568d2917800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:45.785 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:45.782+0000 7f33f48717c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-12T09:28:48.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-12T09:28:48.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:49.056 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:49.054+0000 7efcb4ae27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-12T09:28:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:49.054+0000 7efcb4ae27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-12T09:28:49.057 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:49.054+0000 7efcb4ae27c0 -1 bdev(0x55a2b8af5800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:49.058 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:49.054+0000 7efcb4ae27c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-12T09:28:52.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-12T09:28:52.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:52.350 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:52.346+0000 7f155bb767c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-12T09:28:52.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:52.350+0000 7f155bb767c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-12T09:28:52.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:52.350+0000 7f155bb767c0 -1 bdev(0x55c33a02f800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-12T09:28:52.351 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-12T09:28:52.350+0000 7f155bb767c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-12T09:28:55.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-12T09:28:55.548 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T09:28:55.548 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:55.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-12T09:28:55.564 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:55.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-12T09:28:55.619 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:55.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-12T09:28:55.675 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:55.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-12T09:28:55.732 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:55.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-12T09:28:55.788 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:55.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-12T09:28:55.803 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:55.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-12T09:28:55.859 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:55.860 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-12T09:28:55.915 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:55.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-12T09:28:55.972 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:55.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-12T09:28:56.027 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:28:56.027 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-12T09:28:56.033 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:28:56.033 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-12T09:28:56.076 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T09:28:56.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T09:28:56.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T09:28:56.122 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-12T09:28:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA5tOJmcdxJGBAAa9sFXvdLpELtl6uILi8APA== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA6tOJmB3WKJRAAj8aloHsM5ltPnoV1qLOQqw== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA9tOJmJiHBMxAAX+gdUH6Mkl/0WsvgNNrAxg== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBBtOJmF53PCRAA6DxzvvlHC7Dp3siVA0HqzQ== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBEtOJmFcKZGRAA5MjOq/+wrEdEAHK5zRG7Vg== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA5tOJmlm4KGxAADXtN/n77kQ3UHqnRJlhp4Q== 2024-09-12T09:28:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBKtOJmlENiIRAAZG4/5Ef0LTFTbYWxTxY30A== 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBNtOJmT9DXLhAA9n39HdPC/S5DWVHRWPLoWA== 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBRtOJmYo14AxAAzmn+WZ3N+RPs4BMwWnpHqg== 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBUtOJmhvD9FBAAAnXhfpaidsdcHihpN0ki0A== 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA5tOJm2vH9HRAA/H8v9E5peodFkLagNt1DzQ== 2024-09-12T09:28:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-09-12T09:28:56.125 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQA5tOJm1uehIBAAAcVXavJCezQFVA/9QrpCBg== 2024-09-12T09:28:56.129 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2024-09-12T09:28:56.129 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5tOJmcdxJGBAAa9sFXvdLpELtl6uILi8APA== 2024-09-12T09:28:56.129 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA6tOJmB3WKJRAAj8aloHsM5ltPnoV1qLOQqw== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA9tOJmJiHBMxAAX+gdUH6Mkl/0WsvgNNrAxg== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBBtOJmF53PCRAA6DxzvvlHC7Dp3siVA0HqzQ== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBEtOJmFcKZGRAA5MjOq/+wrEdEAHK5zRG7Vg== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5tOJmlm4KGxAADXtN/n77kQ3UHqnRJlhp4Q== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBKtOJmlENiIRAAZG4/5Ef0LTFTbYWxTxY30A== 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-09-12T09:28:56.130 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBNtOJmT9DXLhAA9n39HdPC/S5DWVHRWPLoWA== 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBRtOJmYo14AxAAzmn+WZ3N+RPs4BMwWnpHqg== 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBUtOJmhvD9FBAAAnXhfpaidsdcHihpN0ki0A== 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5tOJm2vH9HRAA/H8v9E5peodFkLagNt1DzQ== 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2024-09-12T09:28:56.131 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQA5tOJm1uehIBAAAcVXavJCezQFVA/9QrpCBg== 2024-09-12T09:28:56.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T09:28:56.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T09:28:56.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T09:28:56.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T09:28:56.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.609 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T09:28:56.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T09:28:56.954 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T09:28:57.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T09:28:57.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T09:28:57.091 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T09:28:57.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-12T09:28:57.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T09:28:57.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-12T09:28:57.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-12T09:28:57.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T09:28:57.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-12T09:28:57.376 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-12T09:28:57.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T09:28:57.482 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-12T09:28:57.496 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:57.498 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T09:28:57.543 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-12T09:28:57.543 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-12T09:28:57.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-12T09:28:57.546 INFO:tasks.ceph.mon.a:Started 2024-09-12T09:28:57.546 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-12T09:28:57.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-12T09:28:57.548 INFO:tasks.ceph.mon.c:Started 2024-09-12T09:28:57.548 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-12T09:28:57.548 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-12T09:28:57.589 INFO:tasks.ceph.mon.b:Started 2024-09-12T09:28:57.589 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-12T09:28:57.590 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-12T09:28:57.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-12T09:28:57.592 INFO:tasks.ceph.mgr.y:Started 2024-09-12T09:28:57.592 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-12T09:28:57.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-12T09:28:57.594 INFO:tasks.ceph.mgr.x:Started 2024-09-12T09:28:57.595 DEBUG:tasks.ceph:set 0 configs 2024-09-12T09:28:57.595 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T09:28:57.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-12T09:29:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:29:03.930+0000 7fd28a59c640 0 --2- 172.21.15.7:0/1819494826 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd26c006190 0x7fd26c0085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-12T09:29:04.017 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-12T09:29:04.036 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T09:29:04.036 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-12T09:29:04.037 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:29:04.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-12T09:29:04.048 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:29:04.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-12T09:29:04.103 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:29:04.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-12T09:29:04.155 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-12T09:29:04.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-12T09:29:04.211 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:29:04.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-12T09:29:04.224 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:29:04.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-12T09:29:04.278 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:29:04.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-12T09:29:04.330 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-12T09:29:04.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-12T09:29:04.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 37709081-75a3-4326-93b5-5598d715ea86 0 2024-09-12T09:29:04.643 INFO:teuthology.orchestra.run.smithi115.stdout:0 2024-09-12T09:29:04.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new d25b680f-1839-4b35-8ad0-ae6bed56608f 1 2024-09-12T09:29:04.854 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-09-12T09:29:04.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new e209e2d0-0699-4641-a01e-462f5d295d7e 2 2024-09-12T09:29:05.076 INFO:teuthology.orchestra.run.smithi115.stdout:2 2024-09-12T09:29:05.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new f00610a7-05b5-479e-ac16-f9a2b8a4ec1e 3 2024-09-12T09:29:05.279 INFO:teuthology.orchestra.run.smithi115.stdout:3 2024-09-12T09:29:05.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 85af91a4-b30a-47d4-a8cf-4a40e0c18869 4 2024-09-12T09:29:05.481 INFO:teuthology.orchestra.run.smithi115.stdout:4 2024-09-12T09:29:05.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new a11aa028-c5a6-4b28-ad9e-8a24028273cc 5 2024-09-12T09:29:05.697 INFO:teuthology.orchestra.run.smithi115.stdout:5 2024-09-12T09:29:05.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 78df53b3-cb81-420f-8657-bf2c51381f54 6 2024-09-12T09:29:05.949 INFO:teuthology.orchestra.run.smithi115.stdout:6 2024-09-12T09:29:05.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 393284af-2f90-46be-99a0-15f9015c3e7f 7 2024-09-12T09:29:06.063 INFO:tasks.ceph.mgr.x.smithi115.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-12T09:29:06.063 INFO:tasks.ceph.mgr.x.smithi115.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-12T09:29:06.064 INFO:tasks.ceph.mgr.x.smithi115.stderr: from numpy import show_config as show_numpy_config 2024-09-12T09:29:06.160 INFO:teuthology.orchestra.run.smithi115.stdout:7 2024-09-12T09:29:06.176 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T09:29:06.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T09:29:06.179 INFO:tasks.ceph.osd.0:Started 2024-09-12T09:29:06.179 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-12T09:29:06.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-12T09:29:06.181 INFO:tasks.ceph.osd.1:Started 2024-09-12T09:29:06.182 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T09:29:06.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T09:29:06.184 INFO:tasks.ceph.osd.2:Started 2024-09-12T09:29:06.185 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-12T09:29:06.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-12T09:29:06.185 INFO:tasks.ceph.mgr.y.smithi007.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-12T09:29:06.185 INFO:tasks.ceph.mgr.y.smithi007.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-12T09:29:06.185 INFO:tasks.ceph.mgr.y.smithi007.stderr: from numpy import show_config as show_numpy_config 2024-09-12T09:29:06.188 INFO:tasks.ceph.osd.3:Started 2024-09-12T09:29:06.188 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-12T09:29:06.188 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-12T09:29:06.190 INFO:tasks.ceph.osd.4:Started 2024-09-12T09:29:06.190 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-12T09:29:06.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-12T09:29:06.192 INFO:tasks.ceph.osd.5:Started 2024-09-12T09:29:06.192 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-12T09:29:06.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-12T09:29:06.194 INFO:tasks.ceph.osd.6:Started 2024-09-12T09:29:06.194 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-12T09:29:06.194 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-12T09:29:06.196 INFO:tasks.ceph.osd.7:Started 2024-09-12T09:29:06.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:06.409 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:06.409 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:06.157261+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:06.426 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-12T09:29:06.426 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T09:29:07.233 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:07.269 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:07.266+0000 7f2db56a67c0 -1 Falling back to public interface 2024-09-12T09:29:07.376 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:07.374+0000 7f741bb2c7c0 -1 Falling back to public interface 2024-09-12T09:29:07.420 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-12T09:29:07.421 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:06.157261+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:07.435 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T09:29:07.588 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:07.586+0000 7f4fa02c27c0 -1 Falling back to public interface 2024-09-12T09:29:07.613 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:07.610+0000 7f5901f4e7c0 -1 Falling back to public interface 2024-09-12T09:29:07.841 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:07.838+0000 7f3f187117c0 -1 Falling back to public interface 2024-09-12T09:29:07.868 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:07.866+0000 7fdc26f797c0 -1 Falling back to public interface 2024-09-12T09:29:07.885 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:07.882+0000 7fea529bd7c0 -1 Falling back to public interface 2024-09-12T09:29:08.329 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:08.326+0000 7f432798a7c0 -1 Falling back to public interface 2024-09-12T09:29:08.980 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-09-12T09:29:08.978+0000 7fc1e4490640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T09:29:08.982 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-09-12T09:29:08.978+0000 7fc1e4490640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T09:29:11.411 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:11.410+0000 7f2db56a67c0 -1 osd.5 0 log_to_monitors true 2024-09-12T09:29:11.515 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:11.514+0000 7f741bb2c7c0 -1 osd.2 0 log_to_monitors true 2024-09-12T09:29:11.604 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:11.602+0000 7f4fa02c27c0 -1 osd.6 0 log_to_monitors true 2024-09-12T09:29:11.618 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:11.614+0000 7f5901f4e7c0 -1 osd.3 0 log_to_monitors true 2024-09-12T09:29:12.135 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:12.134+0000 7fdc26f797c0 -1 osd.7 0 log_to_monitors true 2024-09-12T09:29:12.149 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:12.146+0000 7f3f187117c0 -1 osd.0 0 log_to_monitors true 2024-09-12T09:29:12.333 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:12.330+0000 7f432798a7c0 -1 osd.4 0 log_to_monitors true 2024-09-12T09:29:12.503 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:12.502+0000 7fea529bd7c0 -1 osd.1 0 log_to_monitors true 2024-09-12T09:29:14.011 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:14.010+0000 7f2db1e61640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T09:29:14.035 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:14.034+0000 7f2da9481640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:14.055 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:14.050+0000 7f4f9ca7d640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T09:29:14.072 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:14.070+0000 7f7417ad5640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T09:29:14.083 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:14.082+0000 7f58fdef7640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T09:29:14.103 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:14.102+0000 7f4f9409d640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:14.108 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:14.106+0000 7f740f907640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:14.124 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:14.122+0000 7f58f5d29640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:14.242 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:14.706 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-12T09:29:14.706 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":13,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:14.045114+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:14.730 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T09:29:14.968 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:14.966+0000 7fea4e966640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T09:29:14.968 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:14.966+0000 7f3f14ecc640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T09:29:14.989 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:14.986+0000 7f3f0c4ec640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:14.989 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:14.986+0000 7fea46798640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:15.060 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:15.058+0000 7f4324145640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T09:29:15.080 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:15.078+0000 7fdc23734640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T09:29:15.095 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:15.094+0000 7f431b765640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:15.131 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:15.130+0000 7fdc1ad54640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:29:21.537 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:21.862 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-12T09:29:21.862 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":17,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:18.095323+0000","last_up_change":"2024-09-12T09:29:16.082807+0000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6817","nonce":381521457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6819","nonce":381521457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6823","nonce":381521457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6821","nonce":381521457}]},"public_addr":"172.21.15.7:6817/381521457","cluster_addr":"172.21.15.7:6819/381521457","heartbeat_back_addr":"172.21.15.7:6823/381521457","heartbeat_front_addr":"172.21.15.7:6821/381521457","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:21.881 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-12T09:29:21.881 INFO:tasks.ceph:Creating RBD pool 2024-09-12T09:29:21.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-12T09:29:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2024-09-12T09:29:22.950 DEBUG:teuthology.orchestra.run.smithi007:> rbd --cluster ceph pool init rbd 2024-09-12T09:29:25.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-12T09:29:25.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-12T09:29:25.985 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T09:29:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:26.327 INFO:teuthology.orchestra.run.smithi007.stdout:[{"version":9,"timestamp":"2024-09-12T09:29:15.081501+0000","name":"","changes":[{"name":"osd.7/osd_mclock_max_capacity_iops_ssd","new_value":"4868.095188"}]}] 2024-09-12T09:29:26.327 INFO:tasks.ceph_manager:config epoch is 9 2024-09-12T09:29:26.327 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T09:29:26.327 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-12T09:29:26.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-12T09:29:26.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:26.711 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1729483949},{"type":"v1","addr":"172.21.15.115:6825","nonce":1729483949}]},"active_addr":"172.21.15.115:6825/1729483949","active_change":"2024-09-12T09:29:07.954482+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":4262105427}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":4137722942}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":478547684}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":950054273}]}]} 2024-09-12T09:29:26.717 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-12T09:29:26.717 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T09:29:26.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:27.033 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:25.943758+0000","last_up_change":"2024-09-12T09:29:16.082807+0000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T09:29:22.198256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6817","nonce":381521457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6819","nonce":381521457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6823","nonce":381521457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6821","nonce":381521457}]},"public_addr":"172.21.15.7:6817/381521457","cluster_addr":"172.21.15.7:6819/381521457","heartbeat_back_addr":"172.21.15.7:6823/381521457","heartbeat_front_addr":"172.21.15.7:6821/381521457","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:27.050 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T09:29:27.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:29:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:27.359 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:29:25.943758+0000","last_up_change":"2024-09-12T09:29:16.082807+0000","last_in_change":"2024-09-12T09:29:06.157261+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T09:29:22.198256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6817","nonce":381521457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6819","nonce":381521457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6823","nonce":381521457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":381521457},{"type":"v1","addr":"172.21.15.7:6821","nonce":381521457}]},"public_addr":"172.21.15.7:6817/381521457","cluster_addr":"172.21.15.7:6819/381521457","heartbeat_back_addr":"172.21.15.7:6823/381521457","heartbeat_front_addr":"172.21.15.7:6821/381521457","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:29:27.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T09:29:27.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T09:29:27.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T09:29:27.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T09:29:27.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T09:29:27.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T09:29:27.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T09:29:27.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T09:29:27.640 INFO:teuthology.orchestra.run.smithi007.stdout:60129542151 2024-09-12T09:29:27.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T09:29:27.648 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-12T09:29:27.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T09:29:27.651 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-12T09:29:27.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T09:29:27.655 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:27.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T09:29:27.656 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:27.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T09:29:27.661 INFO:teuthology.orchestra.run.smithi007.stdout:60129542151 2024-09-12T09:29:27.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T09:29:27.664 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-12T09:29:27.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T09:29:27.668 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:27.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T09:29:28.186 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-12T09:29:28.190 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-12T09:29:28.215 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-12T09:29:28.216 INFO:teuthology.orchestra.run.smithi007.stdout:64424509444 2024-09-12T09:29:28.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.0 2024-09-12T09:29:28.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2024-09-12T09:29:28.227 INFO:teuthology.orchestra.run.smithi007.stdout:64424509443 2024-09-12T09:29:28.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-09-12T09:29:28.239 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-12T09:29:28.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.4 2024-09-12T09:29:28.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.7 2024-09-12T09:29:28.251 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-12T09:29:28.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.2 2024-09-12T09:29:28.274 INFO:teuthology.orchestra.run.smithi007.stdout:60129542146 2024-09-12T09:29:28.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.6 2024-09-12T09:29:28.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.1 2024-09-12T09:29:29.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T09:29:29.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T09:29:29.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T09:29:29.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T09:29:29.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T09:29:29.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T09:29:29.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T09:29:29.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T09:29:29.778 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:29.779 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:29.799 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-12T09:29:29.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-12T09:29:29.807 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-12T09:29:29.807 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.816 INFO:teuthology.orchestra.run.smithi007.stdout:60129542151 2024-09-12T09:29:29.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-09-12T09:29:29.819 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.828 INFO:teuthology.orchestra.run.smithi007.stdout:60129542151 2024-09-12T09:29:29.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-09-12T09:29:29.835 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.837 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-12T09:29:29.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-09-12T09:29:29.849 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-09-12T09:29:29.856 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.873 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-12T09:29:29.881 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-12T09:29:29.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-09-12T09:29:29.892 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-09-12T09:29:29.900 DEBUG:teuthology.parallel:result is None 2024-09-12T09:29:29.900 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T09:29:29.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:30.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:30.229 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:30.245 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T09:29:29.968569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234640,"kb_used_data":1752,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740080,"statfs":{"total":773094113280,"available":772853841920,"internally_reserved":0,"allocated":1794048,"data_stored":1752110,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038612"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.242993+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:26.242993+0000","last_peered":"2024-09-12T09:29:26.242993+0000","last_clean":"2024-09-12T09:29:26.242993+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:26.242993+0000","last_undegraded":"2024-09-12T09:29:26.242993+0000","last_fullsized":"2024-09-12T09:29:26.242993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967463+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:25.967463+0000","last_peered":"2024-09-12T09:29:25.967463+0000","last_clean":"2024-09-12T09:29:25.967463+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:25.967463+0000","last_undegraded":"2024-09-12T09:29:25.967463+0000","last_fullsized":"2024-09-12T09:29:25.967463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045505+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:26.045505+0000","last_peered":"2024-09-12T09:29:26.045505+0000","last_clean":"2024-09-12T09:29:26.045505+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:26.045505+0000","last_undegraded":"2024-09-12T09:29:26.045505+0000","last_fullsized":"2024-09-12T09:29:26.045505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967464+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:25.967464+0000","last_peered":"2024-09-12T09:29:25.967464+0000","last_clean":"2024-09-12T09:29:25.967464+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:25.967464+0000","last_undegraded":"2024-09-12T09:29:25.967464+0000","last_fullsized":"2024-09-12T09:29:25.967464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.966328+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:25.966328+0000","last_peered":"2024-09-12T09:29:25.966328+0000","last_clean":"2024-09-12T09:29:25.966328+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:25.966328+0000","last_undegraded":"2024-09-12T09:29:25.966328+0000","last_fullsized":"2024-09-12T09:29:25.966328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.950437+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:25.950437+0000","last_peered":"2024-09-12T09:29:25.950437+0000","last_clean":"2024-09-12T09:29:25.950437+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:25.950437+0000","last_undegraded":"2024-09-12T09:29:25.950437+0000","last_fullsized":"2024-09-12T09:29:25.950437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045329+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:26.045329+0000","last_peered":"2024-09-12T09:29:26.045329+0000","last_clean":"2024-09-12T09:29:26.045329+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:26.045329+0000","last_undegraded":"2024-09-12T09:29:26.045329+0000","last_fullsized":"2024-09-12T09:29:26.045329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.963671+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:25.963671+0000","last_peered":"2024-09-12T09:29:25.963671+0000","last_clean":"2024-09-12T09:29:25.963671+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:25.963671+0000","last_undegraded":"2024-09-12T09:29:25.963671+0000","last_fullsized":"2024-09-12T09:29:25.963671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.044308+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:26.044308+0000","last_peered":"2024-09-12T09:29:26.044308+0000","last_clean":"2024-09-12T09:29:26.044308+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:26.044308+0000","last_undegraded":"2024-09-12T09:29:26.044308+0000","last_fullsized":"2024-09-12T09:29:26.044308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:30.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:30.542 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:30.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T09:29:29.968569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234640,"kb_used_data":1752,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740080,"statfs":{"total":773094113280,"available":772853841920,"internally_reserved":0,"allocated":1794048,"data_stored":1752110,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038612"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.242993+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:26.242993+0000","last_peered":"2024-09-12T09:29:26.242993+0000","last_clean":"2024-09-12T09:29:26.242993+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:26.242993+0000","last_undegraded":"2024-09-12T09:29:26.242993+0000","last_fullsized":"2024-09-12T09:29:26.242993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967463+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:25.967463+0000","last_peered":"2024-09-12T09:29:25.967463+0000","last_clean":"2024-09-12T09:29:25.967463+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:25.967463+0000","last_undegraded":"2024-09-12T09:29:25.967463+0000","last_fullsized":"2024-09-12T09:29:25.967463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045505+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:26.045505+0000","last_peered":"2024-09-12T09:29:26.045505+0000","last_clean":"2024-09-12T09:29:26.045505+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:26.045505+0000","last_undegraded":"2024-09-12T09:29:26.045505+0000","last_fullsized":"2024-09-12T09:29:26.045505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967464+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:25.967464+0000","last_peered":"2024-09-12T09:29:25.967464+0000","last_clean":"2024-09-12T09:29:25.967464+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:25.967464+0000","last_undegraded":"2024-09-12T09:29:25.967464+0000","last_fullsized":"2024-09-12T09:29:25.967464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.966328+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:25.966328+0000","last_peered":"2024-09-12T09:29:25.966328+0000","last_clean":"2024-09-12T09:29:25.966328+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:25.966328+0000","last_undegraded":"2024-09-12T09:29:25.966328+0000","last_fullsized":"2024-09-12T09:29:25.966328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.950437+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:25.950437+0000","last_peered":"2024-09-12T09:29:25.950437+0000","last_clean":"2024-09-12T09:29:25.950437+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:25.950437+0000","last_undegraded":"2024-09-12T09:29:25.950437+0000","last_fullsized":"2024-09-12T09:29:25.950437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045329+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:26.045329+0000","last_peered":"2024-09-12T09:29:26.045329+0000","last_clean":"2024-09-12T09:29:26.045329+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:26.045329+0000","last_undegraded":"2024-09-12T09:29:26.045329+0000","last_fullsized":"2024-09-12T09:29:26.045329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.963671+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:25.963671+0000","last_peered":"2024-09-12T09:29:25.963671+0000","last_clean":"2024-09-12T09:29:25.963671+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:25.963671+0000","last_undegraded":"2024-09-12T09:29:25.963671+0000","last_fullsized":"2024-09-12T09:29:25.963671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.044308+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:26.044308+0000","last_peered":"2024-09-12T09:29:26.044308+0000","last_clean":"2024-09-12T09:29:26.044308+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:26.044308+0000","last_undegraded":"2024-09-12T09:29:26.044308+0000","last_fullsized":"2024-09-12T09:29:26.044308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:30.560 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T09:29:30.560 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T09:29:30.560 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-12T09:29:30.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-12T09:29:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:30.892 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T09:29:30.909 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-12T09:29:30.910 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-12T09:29:30.921 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-12T09:29:30.921 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': 'd61b08865d4d98738f45ff2e4806d7fb6d03eca1'}} 2024-09-12T09:29:30.921 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-12T09:29:30.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-12T09:29:31.246 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-12T09:29:31.247 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T09:29:31.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:31.542 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:31.558 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T09:29:29.968569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234640,"kb_used_data":1752,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740080,"statfs":{"total":773094113280,"available":772853841920,"internally_reserved":0,"allocated":1794048,"data_stored":1752110,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038612"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.242993+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:26.242993+0000","last_peered":"2024-09-12T09:29:26.242993+0000","last_clean":"2024-09-12T09:29:26.242993+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:26.242993+0000","last_undegraded":"2024-09-12T09:29:26.242993+0000","last_fullsized":"2024-09-12T09:29:26.242993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967463+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:25.967463+0000","last_peered":"2024-09-12T09:29:25.967463+0000","last_clean":"2024-09-12T09:29:25.967463+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:25.967463+0000","last_undegraded":"2024-09-12T09:29:25.967463+0000","last_fullsized":"2024-09-12T09:29:25.967463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045505+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:26.045505+0000","last_peered":"2024-09-12T09:29:26.045505+0000","last_clean":"2024-09-12T09:29:26.045505+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:26.045505+0000","last_undegraded":"2024-09-12T09:29:26.045505+0000","last_fullsized":"2024-09-12T09:29:26.045505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967464+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:25.967464+0000","last_peered":"2024-09-12T09:29:25.967464+0000","last_clean":"2024-09-12T09:29:25.967464+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:25.967464+0000","last_undegraded":"2024-09-12T09:29:25.967464+0000","last_fullsized":"2024-09-12T09:29:25.967464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.966328+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:25.966328+0000","last_peered":"2024-09-12T09:29:25.966328+0000","last_clean":"2024-09-12T09:29:25.966328+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:25.966328+0000","last_undegraded":"2024-09-12T09:29:25.966328+0000","last_fullsized":"2024-09-12T09:29:25.966328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.950437+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:25.950437+0000","last_peered":"2024-09-12T09:29:25.950437+0000","last_clean":"2024-09-12T09:29:25.950437+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:25.950437+0000","last_undegraded":"2024-09-12T09:29:25.950437+0000","last_fullsized":"2024-09-12T09:29:25.950437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045329+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:26.045329+0000","last_peered":"2024-09-12T09:29:26.045329+0000","last_clean":"2024-09-12T09:29:26.045329+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:26.045329+0000","last_undegraded":"2024-09-12T09:29:26.045329+0000","last_fullsized":"2024-09-12T09:29:26.045329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.963671+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:25.963671+0000","last_peered":"2024-09-12T09:29:25.963671+0000","last_clean":"2024-09-12T09:29:25.963671+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:25.963671+0000","last_undegraded":"2024-09-12T09:29:25.963671+0000","last_fullsized":"2024-09-12T09:29:25.963671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.044308+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:26.044308+0000","last_peered":"2024-09-12T09:29:26.044308+0000","last_clean":"2024-09-12T09:29:26.044308+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:26.044308+0000","last_undegraded":"2024-09-12T09:29:26.044308+0000","last_fullsized":"2024-09-12T09:29:26.044308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:31.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:31.844 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:31.860 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T09:29:29.968569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234640,"kb_used_data":1752,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740080,"statfs":{"total":773094113280,"available":772853841920,"internally_reserved":0,"allocated":1794048,"data_stored":1752110,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038612"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.242993+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:26.242993+0000","last_peered":"2024-09-12T09:29:26.242993+0000","last_clean":"2024-09-12T09:29:26.242993+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:26.242993+0000","last_undegraded":"2024-09-12T09:29:26.242993+0000","last_fullsized":"2024-09-12T09:29:26.242993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967463+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:25.967463+0000","last_peered":"2024-09-12T09:29:25.967463+0000","last_clean":"2024-09-12T09:29:25.967463+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:25.967463+0000","last_undegraded":"2024-09-12T09:29:25.967463+0000","last_fullsized":"2024-09-12T09:29:25.967463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045505+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:26.045505+0000","last_peered":"2024-09-12T09:29:26.045505+0000","last_clean":"2024-09-12T09:29:26.045505+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:26.045505+0000","last_undegraded":"2024-09-12T09:29:26.045505+0000","last_fullsized":"2024-09-12T09:29:26.045505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.967464+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:25.967464+0000","last_peered":"2024-09-12T09:29:25.967464+0000","last_clean":"2024-09-12T09:29:25.967464+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:25.967464+0000","last_undegraded":"2024-09-12T09:29:25.967464+0000","last_fullsized":"2024-09-12T09:29:25.967464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.966328+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:25.966328+0000","last_peered":"2024-09-12T09:29:25.966328+0000","last_clean":"2024-09-12T09:29:25.966328+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:25.966328+0000","last_undegraded":"2024-09-12T09:29:25.966328+0000","last_fullsized":"2024-09-12T09:29:25.966328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.950437+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:25.950437+0000","last_peered":"2024-09-12T09:29:25.950437+0000","last_clean":"2024-09-12T09:29:25.950437+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:25.950437+0000","last_undegraded":"2024-09-12T09:29:25.950437+0000","last_fullsized":"2024-09-12T09:29:25.950437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.045329+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:26.045329+0000","last_peered":"2024-09-12T09:29:26.045329+0000","last_clean":"2024-09-12T09:29:26.045329+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:26.045329+0000","last_undegraded":"2024-09-12T09:29:26.045329+0000","last_fullsized":"2024-09-12T09:29:26.045329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:25.963671+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:25.963671+0000","last_peered":"2024-09-12T09:29:25.963671+0000","last_clean":"2024-09-12T09:29:25.963671+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:25.963671+0000","last_undegraded":"2024-09-12T09:29:25.963671+0000","last_fullsized":"2024-09-12T09:29:25.963671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T09:29:26.044308+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:26.044308+0000","last_peered":"2024-09-12T09:29:26.044308+0000","last_clean":"2024-09-12T09:29:26.044308+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:26.044308+0000","last_undegraded":"2024-09-12T09:29:26.044308+0000","last_fullsized":"2024-09-12T09:29:26.044308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":71417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":71436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":266240,"data_stored":661785,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:31.861 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T09:29:31.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 21 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 177484da-1bae-43aa-8977-8d19d52d569f 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-12T09:29:02.966061+0000 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-12T09:29:25.943758+0000 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 5 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-09-12T09:29:32.173 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client jewel 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout: removed_snaps_queue [2~1] 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 8 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/381521457,v1:172.21.15.7:6817/381521457] [v2:172.21.15.7:6818/381521457,v1:172.21.15.7:6819/381521457] exists,up 37709081-75a3-4326-93b5-5598d715ea86 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/2164663219,v1:172.21.15.7:6825/2164663219] [v2:172.21.15.7:6826/2164663219,v1:172.21.15.7:6827/2164663219] exists,up d25b680f-1839-4b35-8ad0-ae6bed56608f 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/139751272,v1:172.21.15.7:6801/139751272] [v2:172.21.15.7:6802/139751272,v1:172.21.15.7:6803/139751272] exists,up e209e2d0-0699-4641-a01e-462f5d295d7e 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/4287683774,v1:172.21.15.7:6809/4287683774] [v2:172.21.15.7:6810/4287683774,v1:172.21.15.7:6811/4287683774] exists,up f00610a7-05b5-479e-ac16-f9a2b8a4ec1e 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6826/1982719715,v1:172.21.15.115:6827/1982719715] [v2:172.21.15.115:6828/1982719715,v1:172.21.15.115:6829/1982719715] exists,up 85af91a4-b30a-47d4-a8cf-4a40e0c18869 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/1321805872,v1:172.21.15.115:6801/1321805872] [v2:172.21.15.115:6802/1321805872,v1:172.21.15.115:6803/1321805872] exists,up a11aa028-c5a6-4b28-ad9e-8a24028273cc 2024-09-12T09:29:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/3633441839,v1:172.21.15.115:6809/3633441839] [v2:172.21.15.115:6810/3633441839,v1:172.21.15.115:6811/3633441839] exists,up 78df53b3-cb81-420f-8657-bf2c51381f54 2024-09-12T09:29:32.175 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/767353918,v1:172.21.15.115:6817/767353918] [v2:172.21.15.115:6818/767353918,v1:172.21.15.115:6819/767353918] exists,up 393284af-2f90-46be-99a0-15f9015c3e7f 2024-09-12T09:29:32.191 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/381521457,v1:172.21.15.7:6817/381521457] [v2:172.21.15.7:6818/381521457,v1:172.21.15.7:6819/381521457] exists,up 37709081-75a3-4326-93b5-5598d715ea86', 'osd.1 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/2164663219,v1:172.21.15.7:6825/2164663219] [v2:172.21.15.7:6826/2164663219,v1:172.21.15.7:6827/2164663219] exists,up d25b680f-1839-4b35-8ad0-ae6bed56608f', 'osd.2 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/139751272,v1:172.21.15.7:6801/139751272] [v2:172.21.15.7:6802/139751272,v1:172.21.15.7:6803/139751272] exists,up e209e2d0-0699-4641-a01e-462f5d295d7e', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/4287683774,v1:172.21.15.7:6809/4287683774] [v2:172.21.15.7:6810/4287683774,v1:172.21.15.7:6811/4287683774] exists,up f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6826/1982719715,v1:172.21.15.115:6827/1982719715] [v2:172.21.15.115:6828/1982719715,v1:172.21.15.115:6829/1982719715] exists,up 85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/1321805872,v1:172.21.15.115:6801/1321805872] [v2:172.21.15.115:6802/1321805872,v1:172.21.15.115:6803/1321805872] exists,up a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'osd.6 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/3633441839,v1:172.21.15.115:6809/3633441839] [v2:172.21.15.115:6810/3633441839,v1:172.21.15.115:6811/3633441839] exists,up 78df53b3-cb81-420f-8657-bf2c51381f54', 'osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/767353918,v1:172.21.15.115:6817/767353918] [v2:172.21.15.115:6818/767353918,v1:172.21.15.115:6819/767353918] exists,up 393284af-2f90-46be-99a0-15f9015c3e7f'] 2024-09-12T09:29:32.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mon.a", 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster": "ceph", 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket_mode": "", 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-12T09:29:32.309 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_client_required": "cephx, none", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_cluster_required": "cephx", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_debug": "true", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_required": "cephx", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_supported": "cephx", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio": "true", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_poll_ms": "250", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_reap_max": "16", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_async_discard_threads": "0", 2024-09-12T09:29:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_block_size": "4096", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_inflight_ios": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_enable_discard": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry": "3", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash": "0", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_hipri": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-12T09:29:32.311 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_type": "", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_alloc_size": "1048576", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_allocator": "hybrid", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_buffered_io": "true", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_for_zeros": "false", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-12T09:29:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_compact_log_sync": "false", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_log_runway": "4194304", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_prefetch": "1048576", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_flush_size": "524288", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_log_runway": "1048576", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery": "false", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_sync_write": "false", 2024-09-12T09:29:32.313 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocation_from_file": "true", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocator": "hybrid", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_async_db_compaction": "true", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi": "true", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-12T09:29:32.314 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_create": "true", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_create": "false", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_path": "", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_size": "0", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_path": "", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_preallocate_file": "false", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_size": "107374182400", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_create": "false", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_path": "", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_size": "100663296", 2024-09-12T09:29:32.315 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs": "true", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune": "true", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-12T09:29:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size": "0", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_type": "2q", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_clone_cow": "true", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-12T09:29:32.317 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_mode": "none", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_csum_type": "crc32c", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_freelist": "false", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-12T09:29:32.318 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_small_allocations": "0", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_read": "true", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_write": "false", 2024-09-12T09:29:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fail_eio": "false", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-12T09:29:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-12T09:29:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_ignore_data_csum": "false", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_op_age": "5.000000", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_alloc_size": "0", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size": "0", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_deferred_txc": "32", 2024-09-12T09:29:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size": "0", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_nid_prealloc": "1024", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_retry_disk_reads": "3", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cf": "true", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-12T09:29:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_mem": "512", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-12T09:29:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_tracing": "false", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-12T09:29:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2": "false", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2_random": "false", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_zero_block_detection": "false", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-12T09:29:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_signatures": "false", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_version": "2", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_signatures": "false", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_version": "2", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_signatures": "false", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_version": "2", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_sign_messages": "true", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "chdir": "", 2024-09-12T09:29:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_acl_type": "", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_asio_thread_count": "2", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_mid": "0.750000", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_size": "16384", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_caps_release_delay": "5", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_check_pool_perm": "true", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_force_sync_read": "false", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_getattr_caps": "false", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_features": "", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_tick_delay": "0", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-12T09:29:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_remount": "false", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_dirsize_rbytes": "true", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_force_lazyio": "false", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_fs": "", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_release_failure": "false", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_inline_size": "4096", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mds_namespace": "", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_metadata": "", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_gid": "-1", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_timeout": "300", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_uid": "-1", 2024-09-12T09:29:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mountpoint": "/", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_notify_timeout": "10", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc": "true", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty": "104857600", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_objects": "1000", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_size": "209715200", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_target_dirty": "8388608", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_permissions": "true", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota": "true", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota_df": "true", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_bytes": "0", 2024-09-12T09:29:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_periods": "4", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_min": "131072", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_reconnect_stale": "false", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_shutdown_timeout": "30", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_snapdir": ".snap", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_tick_interval": "1", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_trace": "", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_try_dentry_invalidate": "false", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_faked_inos": "false", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_random_mds": "false", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog": "false", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_port": "12201", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_monitors": "default=true", 2024-09-12T09:29:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog": "false", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_level": "info", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "-", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network": "", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network_interface": "", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_isal": "false", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_level": "5", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_winsize": "-15", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zstd_level": "1", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-12T09:29:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_op_num_threads": "6", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_stat_interval": "0", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_cpu_cores": "", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_num_threads": "0", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook": "", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook_timeout": "10", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "daemonize": "false", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "daos_pool": "tank", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-12T09:29:32.333 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_alienstore": "0/5", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_allow_any_pool_priority": "false", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok": "1/5", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok_assert_abort": "false", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asserts_on_shutdown": "true", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_auth": "20/20", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bdev": "1/3", 2024-09-12T09:29:32.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluefs": "1/5", 2024-09-12T09:29:32.336 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluestore": "1/5", 2024-09-12T09:29:32.336 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_buffer": "0/1", 2024-09-12T09:29:32.336 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_dedup": "0/5", 2024-09-12T09:29:32.336 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_exporter": "1/5", 2024-09-12T09:29:32.336 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephfs_mirror": "0/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephsqlite": "0/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_client": "0/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_compressor": "1/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_context": "0/1", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crimson_interrupt": "0/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crush": "1/1", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crypto": "1/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cyanstore": "0/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_deliberately_leak_memory": "false", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_disable_randomized_ping": "false", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_dpdk": "1/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_eventtrace": "1/5", 2024-09-12T09:29:32.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filer": "0/1", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filestore": "1/3", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_finisher": "1/1", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_fuse": "1/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeat_testing_span": "0", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeatmap": "1/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_javaclient": "1/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journal": "1/3", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journaler": "0/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_kstore": "1/5", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_lockdep": "0/1", 2024-09-12T09:29:32.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mclock": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_balancer": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_locker": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log_expire": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_migrator": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_quiesce": "3/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_memstore": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgr": "2/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgrc": "1/5", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mon": "20/20", 2024-09-12T09:29:32.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_monc": "0/10", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ms": "1/1", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_none": "0/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objclass": "0/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objectcacher": "0/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objecter": "0/1", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_optracker": "0/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_osd": "1/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_paxos": "20/20", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_perfcounter": "1/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_prioritycache": "1/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rados": "0/5", 2024-09-12T09:29:32.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd": "0/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_mirror": "0/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_pwl": "0/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_replay": "0/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_refs": "0/0", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_reserver": "1/1", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_access": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_datacache": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_dbstore": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_flight": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_notification": "1/5", 2024-09-12T09:29:32.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_sync": "1/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rocksdb": "4/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_backref": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cache": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cleaner": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_device": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_epm": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_journal": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_lba": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_odata": "0/5", 2024-09-12T09:29:32.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_omap": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_onode": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_t": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_tm": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_striper": "0/1", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_test": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_throttle": "1/1", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_timer": "0/1", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_tp": "0/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_trace": "1/5", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "device_failure_prediction_mode": "none", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-12T09:29:32.343 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_graylog": "false", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_journald": "false", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_stderr": "true", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_syslog": "false", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event_tracing": "false", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_addr": "0.0.0.0", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_cert_file": "", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_http_port": "9926", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_key_file": "", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_prio_limit": "5", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-12T09:29:32.344 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sort_metrics": "true", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_stats_period": "5", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "fake_statfs_for_testing": "0", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "fatal_signal_handlers": "true", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_purge_ops": "10", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_truncate_ops": "128", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_apply_finisher_threads": "1", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_blackhole": "false", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_clone_range": "true", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_snap": "true", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_caller_concurrency": "10", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_collect_device_partition_information": "true", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_commit_timeout": "600.000000", 2024-09-12T09:29:32.345 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_inject_read_err": "false", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_omap_check": "false", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_verify_split": "false", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_dump_file": "", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fadvise": "true", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fail_eio": "true", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_shards": "16", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_size": "128", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap": "false", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap_threshold": "4096", 2024-09-12T09:29:32.346 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_inject_stall": "0", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_parallel": "false", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_trailing": "false", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_writeahead": "false", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_kill_at": "0", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs": "0", 2024-09-12T09:29:32.347 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size": "0", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_merge_threshold": "-10", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_odsync_write": "false", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend": "rocksdb", 2024-09-12T09:29:32.348 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend_path": "", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_timeout": "60", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_threads": "2", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_punch_hole": "false", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-12T09:29:32.349 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_ops": "50", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_seek_data_hole": "false", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc": "false", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_splice": "false", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_multiple": "2", 2024-09-12T09:29:32.350 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_rand_factor": "20", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_update_to": "1000", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_enable": "true", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-12T09:29:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_xfs_extsize": "false", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_zfs_snap": "false", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fio_dir": "/tmp/fio", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_allow_other": "true", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_atomic_o_trunc": "true", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_big_writes": "true", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_debug": "false", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_default_permissions": "false", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_disable_pagecache": "false", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_max_write": "0", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_multithreaded": "true", 2024-09-12T09:29:32.352 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_require_active_mds": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_set_user_groups": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_move": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_read": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_write": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_use_invalidate_cb": "true", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_target_name": "ceph", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_file": "", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_inject_failure": "0", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_interval": "5", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "host": "smithi007", 2024-09-12T09:29:32.353 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-12T09:29:32.354 INFO:teuthology.orchestra.run.smithi007.stdout: "inject_early_sigterm": "false", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_agent_port": "6799", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_tracing_enable": "false", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_aio": "true", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_align_min_size": "65536", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_align": "true", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_size": "4096", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_dio": "true", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_discard": "false", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_force_aio": "false", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_ignore_corruption": "false", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_bytes": "10485760", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_entries": "100", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_replay_from": "0", 2024-09-12T09:29:32.355 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_write_header_frequency": "0", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_zero_on_create": "false", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prefetch_periods": "10", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prezero_periods": "5", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_write_head_interval": "15", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "keyfile": "", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-12T09:29:32.356 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_backend": "rocksdb", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_default_stripe_size": "65536", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount": "false", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_bytes": "67108864", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_ops": "512", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_nid_prealloc": "1024", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_onode_map_size": "1024", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_submit_transaction": "false", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_transaction": "false", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "librados_thread_count": "2", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep": "false", 2024-09-12T09:29:32.357 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep_force_backtrace": "false", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_coarse_timestamps": "true", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_flush_on_exit": "false", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_host": "127.0.0.1", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_port": "12201", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_new": "1000", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_recent": "10000", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stderr_prefix": "", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stop_at_utilization": "0.970000", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": "true", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_graylog": "false", 2024-09-12T09:29:32.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_journald": "false", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_stderr": "false", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_syslog": "false", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max_rotating_auth_attempts": "10", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_action_on_write_error": "1", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_alternate_name_max": "8192", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_asio_thread_count": "2", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_export_pin": "true", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_dirs": "true", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-12T09:29:32.359 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_interval": "5", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_interval": "10", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max": "-1", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max_until": "-1", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_merge_size": "50", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_midchunk": "0.300000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_start": "0.200000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_minchunk": "0.001000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_mode": "0", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_max": "1.200000", 2024-09-12T09:29:32.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_min": "0.800000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_overload_epochs": "2", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_bits": "3", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_size": "10000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_target_decay": "10.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_grace": "15.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_interval": "4.000000", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-12T09:29:32.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_mid": "0.700000", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_delay": "0", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_release_free_interval": "10", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_reservation": "0.050000", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_interval": "1", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_threshold": "262144", 2024-09-12T09:29:32.362 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_prealloc_inos": "1000", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_connect_bootstrapping": "false", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_damage_table_max_entries": "10000", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_auth_pins": "false", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_frag": "false", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_scatterstat": "false", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_subtrees": "false", 2024-09-12T09:29:32.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_decay_halflife": "5.000000", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_default_dir_hash": "2", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_defer_session_stale": "true", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_deny_all_reconnect": "false", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_keys_per_op": "16384", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_commit_size": "10", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_entries": "0", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_prefetch": "true", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_on_map": "false", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-12T09:29:32.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_early_reply": "true", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enable_op_tracker": "true", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enforce_unique_name": "true", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random": "true", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-12T09:29:32.365 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_summarize_threshold": "10", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_health_dummy": "false", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_migrator_session_race": "false", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-12T09:29:32.366 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_join_fs": "", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_journal_format": "1", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_create_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_dirfrag_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_export_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_import_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_expire_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_replay_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_link_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_mdstable_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_openc_at": "0", 2024-09-12T09:29:32.367 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_rename_at": "0", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_shutdown_at": "0", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_event_large_threshold": "524288", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_events_per_segment": "1024", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_events": "-1", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_segments": "128", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_pause": "false", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_segment_size": "0", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_threshold": "128", 2024-09-12T09:29:32.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_warn_factor": "2.000000", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_caps_per_client": "1048576", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_flushes": "100000", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_requests": "100000", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_export_size": "20971520", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_file_recover": "32", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_files": "64", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops": "8192", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_snaps_per_dir": "100", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_metrics_update_interval": "2", 2024-09-12T09:29:32.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_per_client": "100", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_working_set": "10000", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_numa_node": "-1", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_complaint_time": "30.000000", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_duration": "600", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_size": "20", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_size": "20", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_log_threshold": "5", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_grace": "15", 2024-09-12T09:29:32.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_interval": "5", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_caps": "30000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_threshold": "262144", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_interval": "1.000000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_gid": "0", 2024-09-12T09:29:32.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_uid": "0", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scrub_stats_review_period": "1", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-12T09:29:32.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_shutdown_check": "0", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_skip_ino": "0", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_max_uid": "4294967294", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_min_uid": "0", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_rstat": "false", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_standby_replay_damaged": "false", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_symlink_recovery": "true", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_exports": "0", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_fragments": "0", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_tick_interval": "5.000000", 2024-09-12T09:29:32.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_valgrind_exit": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_backtrace": "1", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_scatter": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_sessions": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool_debug": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_device_bytes": "1073741824", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_set": "false", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_size": "65536", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_bytes": "134217728", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_messages": "512", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-12T09:29:32.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_disabled_modules": "", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_enable_op_tracker": "true", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_creating": "1024", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_num_change": "128", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_bytes": "134217728", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_messages": "128", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_bytes": "134217728", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_messages": "128", 2024-09-12T09:29:32.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_duration": "600", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_size": "20", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_log_threshold": "5", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_bytes": "536870912", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_messages": "8192", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_pool": "true", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_standby_modules": "true", 2024-09-12T09:29:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_period": "5", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_threshold": "5", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_tick_period": "2", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_delete": "true", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_size_one": "true", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_auth_validate_all_caps": "true", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_bytes": "104857600", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_directed_command_retry": "5", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-12T09:29:32.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_parallel": "3", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_log_interval": "1.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_interval": "10.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_target_rank": "-1", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_level": "debug", 2024-09-12T09:29:32.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_file": "true", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_journald": "false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_bootstrap": "false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_start": "false", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_trim": "true", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-12T09:29:32.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cpu_threads": "4", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_crush_min_required_version": "hammer", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_daemon_bytes": "419430400", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_crit": "5", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_warn": "5", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_size_warn": "16106127360", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_json": "false", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_transactions": "false", 2024-09-12T09:29:32.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_extra_checks": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_reef": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_squid": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_mkfs_grace": "120", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_uptime_grace": "60", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_default_strategy": "1", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_timeout": "5.000000", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-12T09:29:32.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_divisor": "2", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_enable_op_tracker": "true", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fake_pool_delete": "false", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_force_quorum_join": "false", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_globalid_prealloc": "10000", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_detail_to_clog": "false", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_log_update_period": "5", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_max_detail": "50", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog": "true", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_interval": "600", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-12T09:29:32.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host": "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.115", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host_override": "", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_initial_members": "", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease": "5.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_full_interval": "50", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max": "10000", 2024-09-12T09:29:32.383 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max_summary": "50", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_epochs": "500", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_osd": "10000", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pg_per_osd": "10000", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pool_pg_num": "65536", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_force_trim_to": "0", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_skip_sanity": "false", 2024-09-12T09:29:32.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_autotune": "true", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_target": "2147483648", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_beacon_grace": "30", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_digest_period": "5", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_inactive_grace": "60", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_min_osdmap_epochs": "500", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_complaint_time": "30", 2024-09-12T09:29:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_duration": "600", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_size": "20", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_size": "20", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_log_threshold": "5", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_in": "false", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-12T09:29:32.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size": "500", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_interval": "600", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_force_trim_to": "0", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-12T09:29:32.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_down_reporters": "2", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_report_timeout": "900", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-12T09:29:32.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_stuck_threshold": "60", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_probe_timeout": "2.000000", 2024-09-12T09:29:32.389 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_change": "0.050000", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_osds": "4", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_interval": "86400", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_max_keys": "100", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_timeout": "300", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_session_timeout": "300", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_smart_report_timeout": "5", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stat_smooth_intervals": "6", 2024-09-12T09:29:32.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_min_size": "2", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_size": "4", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_debug": "false", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_provider_kill_at": "0", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_requester_kill_at": "0", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_timeout": "60.000000", 2024-09-12T09:29:32.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_target_pg_per_osd": "100", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_tick_interval": "5", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_interval": "300.000000", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_use_min_delay_socket": "false", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_older_version_delay": "604800", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-12T09:29:32.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_misplaced": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_older_version": "true", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": "", 2024-09-12T09:29:32.393 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_tracing_enabled": "false", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_op_threads": "3", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_cm": "false", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_device_name": "", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_dscp": "96", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-12T09:29:32.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_local_gid": "", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_port_num": "1", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_sl": "3", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_support_srq": "true", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_type": "ib", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_reap_threshold": "5", 2024-09-12T09:29:32.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_before_connect": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv4": "true", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv6": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr1": "true", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr2": "true", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_max": "7568", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_min": "6800", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_count": "3", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_delay": "5", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_client": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mds": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mgr": "false", 2024-09-12T09:29:32.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mon": "false", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_osd": "false", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_mode": "crc secure", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_mode": "crc secure", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_type": "", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_compress_secure": "false", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_idle_timeout": "900", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_ready_timeout": "10", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_data": "true", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_header": "true", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bad_msg": "false", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bug": "true", 2024-09-12T09:29:32.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_old_message": "true", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_skipped_message": "false", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_coremask": "0xF", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_enable_tso": "true", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hugepages": "", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-12T09:29:32.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_lro": "true", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_memory_channel": "4", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_pmd": "", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_port_id": "0", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_on_send": "false", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_initial_backoff": "0.200000", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_max": "1.000000", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-12T09:29:32.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_type": "", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_network_congestion": "0", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_socket_failures": "5000", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_learn_addr_from_peer": "true", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_accept_failures": "4", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_backoff": "15.000000", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_client_mode": "secure crc", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_service_mode": "secure crc", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_min_size": "1024", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_mode": "none", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-12T09:29:32.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_min_cost": "65536", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_public_type": "", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_service_mode": "crc secure", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_listen_backlog": "512", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_nodelay": "true", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_rcvbuf": "0", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_type": "async+posix", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "no_config_file": "false", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_completion_locks_per_session": "32", 2024-09-12T09:29:32.401 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_ops": "1024", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_tick_interval": "5.000000", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_timeout": "10.000000", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_blackhole": "false", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "openssl_engine_opts": "", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_delay_time": "5.000000", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_hist_halflife": "1000", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_low_ops": "2", 2024-09-12T09:29:32.402 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_ops": "4", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_slop": "0.020000", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_asio_thread_count": "2", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_async_recovery_min_cost": "100", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_max": "512", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_min": "64", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_degraded": "false", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_peering": "false", 2024-09-12T09:29:32.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_unfound": "true", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_beacon_report_interval": "300", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_duration": "30", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_max_block_size": "67108864", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blkin_trace_all": "false", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_for_log_corruption": "false", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-12T09:29:32.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_update_on_start": "true", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_cap": "256", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_size_cap": "524288000", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_op_priority": "63", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_watch_timeout": "30", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_command_max_records": "256", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_compact_on_start": "false", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-12T09:29:32.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_on_start": "true", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_weight_set": "true", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_feed_pullee": "-1", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_misdirected_ops": "false", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_acting_change": "false", 2024-09-12T09:29:32.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_purge_strays": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_op_order": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_shutdown": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_trim_objects": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_snaps": "false", 2024-09-12T09:29:32.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_keys": "1024", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_stride": "524288", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_notify_timeout": "30", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep": "0.000000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-12T09:29:32.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_discard_disconnected_ops": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ec_partial_reads": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_enable_op_tracker": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_info": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-12T09:29:32.409 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_function_tracing": "false", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_grace": "20", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_interval": "6", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_peers": "10", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_size": "2000", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_stale": "600", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_max_size": "100000", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_min_size": "1000", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-12T09:29:32.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_size": "5120", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_kill_backfill_at": "0", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_cache_size": "50", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_dedup": "true", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max": "40", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_share_max_epochs": "40", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_name_len": "100", 2024-09-12T09:29:32.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_size": "0", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_backfills": "1", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_count": "5", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_period": "600", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_name_len": "2048", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_namespace_len": "256", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_size": "134217728", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_blocked_by": "16", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_log_entries": "10000", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pgls": "1024", 2024-09-12T09:29:32.412 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_cost": "8388608", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_objects": "10", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_scrubs": "3", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_trimming_pgs": "2", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_op_reply_len": "64", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_size": "90", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-12T09:29:32.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_profile": "balanced", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-12T09:29:32.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_base": "805306368", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_min": "134217728", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target": "4294967296", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_autotune": "false", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_pg_log_entries": "250", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_recovery_priority": "0", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_interval": "5", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-12T09:29:32.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_cache_shards": "32", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_op_tracker_shard": "32", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_auto_affinity": "true", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_node": "-1", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_prefer_iface": "true", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objecter_finishers": "1", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore": "bluestore", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_fuse": "false", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_tracing": "false", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_complaint_time": "30.000000", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_duration": "600", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_size": "20", 2024-09-12T09:29:32.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_size": "20", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_log_threshold": "5", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards": "0", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_hdd": "5", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_ssd": "8", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_min_cost": "65536", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue_cut_off": "high", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-12T09:29:32.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_timeout": "15", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_open_classes_on_start": "true", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_os_flags": "0", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_peering_op_priority": "255", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_cost": "1048576", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_priority": "5", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_max": "10000", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_min": "100", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-12T09:29:32.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crimson": "false", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-12T09:29:32.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flags": "0", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_min_size": "0", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_num": "32", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pgp_num": "0", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_ratio": "70", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_size": "2", 2024-09-12T09:29:32.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_type": "replicated", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_push_per_object_cost": "1000", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap": "true", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_cost": "20971520", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active": "0", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-12T09:29:32.421 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_single_start": "1", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_priority": "3", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_priority": "5", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep": "0.000000", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_repair_during_recovery": "false", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_requested_scrub_priority": "120", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-12T09:29:32.422 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair": "false", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_hour": "0", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_week_day": "0", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_max": "25", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_min": "5", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_cost": "52428800", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_during_recovery": "false", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_hour": "0", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_week_day": "0", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_event_cost": "4096", 2024-09-12T09:29:32.423 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_invalid_stats": "true", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_preemptions": "5", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_priority": "5", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_delay": "30", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_retry_trimming": "10", 2024-09-12T09:29:32.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_sleep": "0.000000", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_data_digest": "false", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_smart_report_timeout": "5", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_cost": "1048576", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_priority": "5", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-12T09:29:32.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_transaction_size": "30", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-12T09:29:32.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tracing": "false", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_use_stale_snap": "false", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "osdc_blkin_trace_all": "false", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_kill_at": "0", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_max_join_drift": "10", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min": "500", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min_wait": "0.050000", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_propose_interval": "1.000000", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max": "500", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_min": "250", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_stash_full_interval": "25", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_max": "500", 2024-09-12T09:29:32.427 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_min": "250", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "perf": "true", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "-", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrv": "", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "public_bind_addr": "-", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network": "", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network_interface": "", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_busy_polling": "false", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_enabled": "false", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_session_max_number": "256", 2024-09-12T09:29:32.428 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_mon_op_timeout": "0", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_osd_op_timeout": "0", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_tracing": "false", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_atime_update_interval": "60", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_parent_reads": "false", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_snap_reads": "false", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blkin_trace_all": "false", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache": "true", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-12T09:29:32.429 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_policy": "writearound", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_size": "33554432", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_clone_copy_on_read": "false", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_compression_hint": "none", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_concurrent_management_ops": "10", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_clone_format": "auto", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_data_pool": "", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_features": "61", 2024-09-12T09:29:32.430 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_format": "2", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_map_options": "", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_order": "22", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_pool": "rbd", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_count": "0", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_unit": "0", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_enable_alloc_hint": "true", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler": "simple", 2024-09-12T09:29:32.431 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_order": "24", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_pool": "", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_splay_width": "4", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_parent_reads": "false", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_snap_reads": "false", 2024-09-12T09:29:32.432 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-12T09:29:32.433 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-12T09:29:32.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mtime_update_interval": "60", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_non_blocking_aio": "true", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_thread_timeout": "60", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_threads": "1", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_parent_cache_enabled": "false", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-12T09:29:32.435 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_plugins": "", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_limit": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_exclude_ops": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_limit": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-12T09:29:32.436 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_read_from_replica_policy": "default", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-12T09:29:32.437 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_skip_partial_discard": "true", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_tracing": "false", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_names": "true", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_pool": "true", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_base_url": "", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_log_level": "", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_acl_grants_max_num": "100", 2024-09-12T09:29:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_admin_entry": "admin", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_asio_assert_yielding": "false", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_backend_store": "rados", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_barbican_url": "", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_beast_enable_async": "true", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache": "false", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-12T09:29:32.439 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_enabled": "true", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_expiry_interval": "900", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_lru_size": "25000", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_config_store": "rados", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_content_length_compat": "false", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress": "true", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cors_rules_max_num": "100", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cross_domain_policy": "", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_addr": "", 2024-09-12T09:29:32.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_password": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_username": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_require_ssl": "true", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-12T09:29:32.441 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_addr": "", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_auth": "token", 2024-09-12T09:29:32.442 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_namespace": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_prefix": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_token_file": "", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_buffersize": "524288", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_time": "300", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-12T09:29:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-12T09:29:32.444 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_changes_size": "1000", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_num_shards": "128", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_window": "30", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-12T09:29:32.445 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_disable_s3select": "false", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-12T09:29:32.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_name": "", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_s3website_name": "", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding": "true", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-12T09:29:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_gc_threads": "true", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_lc_threads": "true", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_mdsearch": "true", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_ops_log": "false", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_quota_threads": "true", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_static_website": "false", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_usage_log": "false", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enforce_swift_acls": "true", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_exit_timeout_secs": "120", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_expose_bucket": "false", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_extended_http_attrs": "", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_filter": "none", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-12T09:29:32.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontends": "beast port=7480", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred": "50", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_objs": "32", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_period": "3600", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-12T09:29:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_init_timeout": "300", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_connection_idle": "300", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_domain": "", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password": "", 2024-09-12T09:29:32.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password_path": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_project": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_tenant": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token_path": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_user": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_domain": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_password": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_project": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_user": "", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-12T09:29:32.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_url": "", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_debug_interval": "-1", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_lock_max_time": "90", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_objs": "32", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_rules": "1000", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_worker": "3", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_thread_delay": "0", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_dnattr": "uid", 2024-09-12T09:29:32.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchfilter": "", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_uri": "", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_http_headers": "", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name_utc": "false", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-12T09:29:32.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_name_len": "0", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_size": "0", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_chunk_size": "4194304", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_listing_results": "5000", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_notify_retries": "10", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_param_size": "1048576", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_size": "5368709120", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_slo_entries": "1000", 2024-09-12T09:29:32.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_log_max_shards": "64", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mp_lock_max_time": "600", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-12T09:29:32.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_max_gc": "300", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_async_rados_threads": "32", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_control_oids": "8", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_numa_node": "-1", 2024-09-12T09:29:32.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_chunk_size": "100", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_gc_interval": "600", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_timeout": "600", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_tracing": "false", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_token": "", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_url": "", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_verify_ssl": "true", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-12T09:29:32.457 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_rados": "false", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_socket_path": "", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval": "2.000000", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lanes": "3", 2024-09-12T09:29:32.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_partitions": "3", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_continue": "true", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_tracing": "false", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm": "", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_id": "", 2024-09-12T09:29:32.459 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region": "", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_request_uri": "", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_batch_size": "64", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_max_aio": "128", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_num_logs": "16", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-12T09:29:32.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_thread_interval": "600", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_resolve_cname": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_run_sync_thread": "true", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-12T09:29:32.461 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_scheduler_type": "throttler", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_script_uri": "", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_service_provider_name": "", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_id": "", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_secret": "", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_entry": "sts", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_key": "sts", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_min_session_duration": "900", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_token_introspection_url": "", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_account_in_url": "false", 2024-09-12T09:29:32.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_entry": "auth", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_url": "", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_custom_header": "", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_need_stats": "true", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_tenant_name": "", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_token_expiration": "86400", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url": "", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url_prefix": "swift", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_lease_period": "120", 2024-09-12T09:29:32.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_thread_pool_size": "512", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-12T09:29:32.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_comment": "", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_createby": "", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_encoding": "", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_flag": "false", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_origin": "", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_tracker": "", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_trust_forwarded_https": "false", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_shards": "32", 2024-09-12T09:29:32.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_user_shards": "1", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_use_opa_authz": "false", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache": "false", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_max_buckets": "1000", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_policies_max_num": "100", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_unique_email": "true", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_verify_ssl": "true", 2024-09-12T09:29:32.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone": "", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_id": "", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup": "", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_id": "", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_auto_session_clear": "true", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_block_size": "4096", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-12T09:29:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_size": "536870912", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_index_type": "binary_search", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-12T09:29:32.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_partition_filters": "false", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_perf": "false", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "run_dir": "/var/run/ceph", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_block_create": "true", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cache_lru_size": "67108864", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_max_object_size": "16777216", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_device_size": "53687091200", 2024-09-12T09:29:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_full_integrity_check": "false", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_capacity": "16", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_main_device_type": "SSD", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-12T09:29:32.470 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_segment_size": "67108864", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "set_keepcaps": "false", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "setgroup": "", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser": "", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser_match_path": "", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "thp": "false", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_default_timeout": "60", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "throttler_perf_counter": "true", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_dir": "/tmp", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_compressor_enabled": "false", 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-12T09:29:32.471 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:32.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-12T09:29:32.619 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-12T09:29:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:29:32.632 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-12T09:29:32.632 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:29:32.643 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-12T09:29:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:29:32.660 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-12T09:29:32.664 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T09:29:32.664 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-09-12T09:29:32.669 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T09:29:32.670 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-12T09:29:32.681 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T09:29:32.681 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T09:29:32.682 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-12T09:29:32.682 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-12T09:29:32.682 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-12T09:29:32.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:32.683 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-12T09:29:32.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:32.683 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-12T09:29:32.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:29:32.684 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-12T09:29:32.708 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:32.706+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:32.724 INFO:teuthology.orchestra.run.smithi007.stdout:mapped writes DISABLED 2024-09-12T09:29:32.724 INFO:teuthology.orchestra.run.smithi007.stdout:mapped reads DISABLED 2024-09-12T09:29:32.724 INFO:teuthology.orchestra.run.smithi007.stdout:Seed set to 3422 2024-09-12T09:29:32.785 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:32.782+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:32.824 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:32.824 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:32.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:32.840 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:32.841 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:32.841 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T09:29:32.841 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:32.857 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:32.858 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:32.874 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:32.886 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:32.882+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:32.890 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:32.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:32.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:32.929 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:32.949 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:32.949 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:32.952 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:32.952 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:32.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:32.969 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:32.970 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:32.986 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:32.982+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:33.080 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.080 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.087 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:33.086+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:33.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:33.188 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:33.186+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:33.289 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:33.286+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:33.390 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:33.390+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:33.397 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.397 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:33.491 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:33.490+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:33.535 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.535 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:33.596 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:33.594+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:33.640 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:29:33.676 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.677 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:33.693 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:33.690+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:33.795 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:33.794+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:33.855 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.856 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:33.895 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:33.894+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:33.993 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:33.994 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:33.996 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:33.994+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:34.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:34.097 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:34.094+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:34.198 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:34.194+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:34.299 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:34.298+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:34.401 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:34.398+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:34.459 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.460 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:34.502 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:34.498+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:34.603 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:34.602+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:34.605 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.605 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:34.715 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:34.714+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.793 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.805 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:34.802+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:34.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:34.906 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:34.902+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:34.971 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:34.971 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:34.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:34.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:35.035 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:35.034+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:35.107 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:35.106+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:35.145 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:35.151 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:35.161 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:35.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:35.178 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:35.196 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:35.207 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:35.206+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:35.214 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:35.234 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:35.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:35.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:35.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:35.308 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:35.306+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:35.327 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:35.327 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:35.341 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:35.408 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:35.406+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:35.507 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:35.507 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:35.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4217.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:23.938243+0000", 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.569494052, 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011582804, 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4217", 2024-09-12T09:29:35.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/4102530975", 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938243+0000", 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938243+0000", 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938254+0000", 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1457000000000001e-05 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938269+0000", 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4047e-05 2024-09-12T09:29:35.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938293+0000", 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3986e-05 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938316+0000", 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000001e-05 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938507+0000", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019084500000000001 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.938526+0000", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.939672+0000", 2024-09-12T09:29:35.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011457990000000001 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.939693+0000", 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.105e-05 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.943343+0000", 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036503270000000001 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.943741+0000", 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039707499999999999 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.944132+0000", 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000391467 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:35.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945891+0000", 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017588289999999999 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.949634+0000", 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037427900000000002 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.949764+0000", 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013018900000000001 2024-09-12T09:29:35.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.949826+0000", 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2036e-05 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4217.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:23.945463+0000", 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.562273597000001, 2024-09-12T09:29:35.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000442044, 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945463+0000", 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945463+0000", 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945465+0000", 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7999999999999999e-06 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945476+0000", 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0837e-05 2024-09-12T09:29:35.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945510+0000", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3503000000000001e-05 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945535+0000", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5199e-05 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945857+0000", 2024-09-12T09:29:35.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032239200000000001 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945872+0000", 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4895999999999999e-05 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.945906+0000", 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3417000000000002e-05 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4217.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:23.954392+0000", 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.553344793000001, 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039570499999999999, 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4217", 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/4102530975", 2024-09-12T09:29:35.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954392+0000", 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954392+0000", 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954395+0000", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954408+0000", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2964e-05 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954437+0000", 2024-09-12T09:29:35.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8693000000000001e-05 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954452+0000", 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5543e-05 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954575+0000", 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122223 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.954788+0000", 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021346200000000001 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4217.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:23.955305+0000", 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.552432475, 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033878400000000001, 2024-09-12T09:29:35.525 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4217", 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/4102530975", 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955305+0000", 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955305+0000", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955307+0000", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-06 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955315+0000", 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9629999999999998e-06 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955336+0000", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0962999999999999e-05 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955355+0000", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9924000000000001e-05 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955453+0000", 2024-09-12T09:29:35.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7589000000000004e-05 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:23.955643+0000", 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019034499999999999 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4217.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:24.952453+0000", 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.555284254, 2024-09-12T09:29:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074942230000000004, 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4217", 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/4102530975", 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952453+0000", 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952453+0000", 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952456+0000", 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7099999999999999e-06 2024-09-12T09:29:35.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952470+0000", 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4355999999999999e-05 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952502+0000", 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1664e-05 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952519+0000", 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7618999999999999e-05 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952656+0000", 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013700299999999999 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.952684+0000", 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7339999999999999e-05 2024-09-12T09:29:35.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.956443+0000", 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037592490000000001 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.956653+0000", 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000209922 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:29:35.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.956989+0000", 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033627800000000002 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.958550+0000", 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001560737 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959751+0000", 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012014059999999999 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959835+0000", 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3503000000000003e-05 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959947+0000", 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011243600000000001 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4217.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-12T09:29:35.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:24.959559+0000", 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.548177908, 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000402433, 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959559+0000", 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959559+0000", 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959561+0000", 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8759999999999999e-06 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959570+0000", 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2799999999999992e-06 2024-09-12T09:29:35.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959592+0000", 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1733e-05 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959605+0000", 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3267e-05 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959723+0000", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117893 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959743+0000", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9589000000000002e-05 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:24.959962+0000", 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021879500000000001 2024-09-12T09:29:35.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4217.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:25.954408+0000", 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5533295299999992, 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011996022, 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4217", 2024-09-12T09:29:35.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/4102530975", 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954408+0000", 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954408+0000", 2024-09-12T09:29:35.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954411+0000", 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1870000000000001e-06 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954426+0000", 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4803e-05 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954461+0000", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5500000000000002e-05 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954518+0000", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6999000000000001e-05 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954723+0000", 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020530900000000001 2024-09-12T09:29:35.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.954734+0000", 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0903e-05 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.963072+0000", 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083377840000000009 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.963205+0000", 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013295000000000001 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.963412+0000", 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020704499999999999 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.965228+0000", 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018162650000000001 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966239+0000", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001010298 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966324+0000", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5715000000000002e-05 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966404+0000", 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9264000000000005e-05 2024-09-12T09:29:35.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4217.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:25.966025+0000", 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5417116100000001, 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038798800000000003, 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966025+0000", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966025+0000", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966027+0000", 2024-09-12T09:29:35.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0040000000000002e-06 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966037+0000", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2629999999999992e-06 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966063+0000", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6400000000000001e-05 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966078+0000", 2024-09-12T09:29:35.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4620000000000001e-05 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966210+0000", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131886 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966229+0000", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9715999999999999e-05 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:25.966413+0000", 2024-09-12T09:29:35.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018409899999999999 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:35.552 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:35.553 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:35.554 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:35.550+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:35.654 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:35.654+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:35.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:29:35.722 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:29:35.723 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:35.737 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:35.787 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:35.786+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:35.856 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:35.854+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:35.863 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:35.863 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:35.863 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:35.863 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:35.863 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:35.877 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:35.963 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:35.962+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:36.041 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:36.041 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:36.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:36.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-09-12T09:29:36.041 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:36.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:36.099 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:36.098+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:36.159 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:36.154+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:36.221 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:36.221 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:36.221 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:29:36.221 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:36.235 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:36.279 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:36.278+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:36.359 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:36.360 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:36.358+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:36.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:36.461 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:36.462+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:36.536 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.391304028, 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090235599999999999, 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142641+0000", 2024-09-12T09:29:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7600000000000006e-07 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142791+0000", 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014992299999999999 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142808+0000", 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6946e-05 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142818+0000", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6870000000000006e-06 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146487+0000", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036697040000000002 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146550+0000", 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3001000000000002e-05 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146656+0000", 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010525 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.148650+0000", 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019946899999999999 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151520+0000", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028695980000000001 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151608+0000", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8269000000000005e-05 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151664+0000", 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5516000000000002e-05 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290927826000001, 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097954749999999997, 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-12T09:29:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243019+0000", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5830000000000001e-06 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243032+0000", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3273e-05 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243059+0000", 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6466999999999999e-05 2024-09-12T09:29:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243074+0000", 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5780000000000001e-05 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248093+0000", 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050182890000000004 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248239+0000", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014628199999999999 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248459+0000", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220125 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252338+0000", 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038786530000000001 2024-09-12T09:29:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252513+0000", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017509900000000001 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252680+0000", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167135 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252812+0000", 2024-09-12T09:29:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013178899999999999 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290831901000001, 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013226388, 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243113+0000", 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243190+0000", 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7043000000000001e-05 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243204+0000", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3587000000000001e-05 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243219+0000", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4943000000000001e-05 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249826+0000", 2024-09-12T09:29:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006606776 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249962+0000", 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013611499999999999 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.250284+0000", 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032211799999999998 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253909+0000", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036251310000000002 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256018+0000", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021093829999999998 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256216+0000", 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019755900000000001 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256339+0000", 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012262899999999999 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290672738000001, 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013432342, 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243272+0000", 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6599999999999994e-07 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243281+0000", 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8240000000000006e-06 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243295+0000", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.377e-05 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243303+0000", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4360000000000002e-06 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251352+0000", 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080488069999999998 2024-09-12T09:29:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251497+0000", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014492199999999999 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251707+0000", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000209946 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256473+0000", 2024-09-12T09:29:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047662 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256494+0000", 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256593+0000", 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8301999999999995e-05 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256704+0000", 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111079 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290624234999999, 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014730705, 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 46 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243321+0000", 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.47e-07 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243329+0000", 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8059999999999995e-06 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243338+0000", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8400000000000007e-06 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243346+0000", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5440000000000001e-06 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252951+0000", 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096052099999999994 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253091+0000", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013976900000000001 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253286+0000", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019547499999999999 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256852+0000", 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035660039999999998 2024-09-12T09:29:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257859+0000", 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010067539999999999 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257936+0000", 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7156999999999993e-05 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258051+0000", 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114299 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290569144999999, 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015106299, 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 47 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243376+0000", 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3300000000000001e-07 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243454+0000", 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8106000000000001e-05 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243468+0000", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.375e-05 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243476+0000", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6230000000000003e-06 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253981+0000", 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010505067999999999 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254222+0000", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000240122 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254684+0000", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046238699999999998 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258069+0000", 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033854919999999999 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258205+0000", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013541900000000001 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258377+0000", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017185900000000001 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258481+0000", 2024-09-12T09:29:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010464 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290450486000001, 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018308963000000001, 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 48 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243495+0000", 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9999999999999996e-07 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243501+0000", 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-06 2024-09-12T09:29:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243518+0000", 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7073000000000001e-05 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243526+0000", 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7770000000000002e-06 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256954+0000", 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013428383 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257083+0000", 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892900000000001 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257284+0000", 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020067500000000001 2024-09-12T09:29:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261519+0000", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042348539999999997 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261670+0000", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015173899999999999 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261742+0000", 2024-09-12T09:29:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1476000000000005e-05 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261803+0000", 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0956999999999999e-05 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290328983999999, 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018360113000000001, 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243617+0000", 2024-09-12T09:29:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.657e-06 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243629+0000", 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.242e-05 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243650+0000", 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243665+0000", 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4163e-05 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258596+0000", 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014931444 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258819+0000", 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022252799999999999 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.259193+0000", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037449800000000002 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261650+0000", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024574620000000001 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261818+0000", 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016791500000000001 2024-09-12T09:29:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261924+0000", 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105213 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261975+0000", 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1722999999999999e-05 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.290196301000002, 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020862525999999999, 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243750+0000", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.626e-06 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243917+0000", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167193 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243947+0000", 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0145999999999999e-05 2024-09-12T09:29:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243969+0000", 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1654e-05 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260058+0000", 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016089882999999999 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260195+0000", 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013658600000000001 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260404+0000", 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020855900000000001 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264456+0000", 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040527699999999998 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264476+0000", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9823999999999999e-05 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264547+0000", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1116000000000001e-05 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264610+0000", 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3169000000000005e-05 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.289939569000001, 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020895776000000001, 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244006+0000", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5770000000000001e-06 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244018+0000", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1636e-05 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244037+0000", 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9360000000000001e-05 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244074+0000", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6303000000000001e-05 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262033+0000", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017959040999999999 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262203+0000", 2024-09-12T09:29:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017056300000000001 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262414+0000", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021082800000000001 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264629+0000", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002215416 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264706+0000", 2024-09-12T09:29:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6667000000000003e-05 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264843+0000", 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136852 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264900+0000", 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7533e-05 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.289796670000001, 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021340904000000001, 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244149+0000", 2024-09-12T09:29:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5629999999999999e-06 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244161+0000", 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2106999999999999e-05 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244191+0000", 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0162999999999999e-05 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244211+0000", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.995e-05 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263126+0000", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018914665000000001 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263373+0000", 2024-09-12T09:29:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024669200000000002 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263771+0000", 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039814700000000001 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265200+0000", 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001428728 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265241+0000", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1350000000000002e-05 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265382+0000", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141209 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265488+0000", 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010632999999999999 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.267362837, 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010003580999999999, 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 56 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266583+0000", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5239999999999999e-06 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266708+0000", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012525899999999999 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266730+0000", 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2206000000000001e-05 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266747+0000", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509999999999999e-05 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.269973+0000", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032259889999999999 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.270123+0000", 2024-09-12T09:29:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014996000000000001 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.270303+0000", 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180318 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.272397+0000", 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020935709999999998 2024-09-12T09:29:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276353+0000", 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039558589999999999 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276482+0000", 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012954199999999999 2024-09-12T09:29:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276585+0000", 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102843 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.029524296999998, 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0095335880000000008, 2024-09-12T09:29:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 74 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504421+0000", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3e-07 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504444+0000", 2024-09-12T09:29:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2979000000000001e-05 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504461+0000", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7520000000000002e-05 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504469+0000", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.588e-06 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.511828+0000", 2024-09-12T09:29:36.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073590529999999999 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.511902+0000", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4140999999999994e-05 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512004+0000", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010264600000000001 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513829+0000", 2024-09-12T09:29:36.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018242200000000001 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513879+0000", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9863e-05 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513919+0000", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0382000000000001e-05 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513953+0000", 2024-09-12T09:29:36.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4566000000000001e-05 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.029458937000001, 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099651919999999995, 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 75 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504486+0000", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504605+0000", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118882 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504634+0000", 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9141000000000001e-05 2024-09-12T09:29:36.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504647+0000", 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2661e-05 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512339+0000", 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076925769999999999 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512404+0000", 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4318000000000003e-05 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512503+0000", 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9037999999999996e-05 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513842+0000", 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001339886 2024-09-12T09:29:36.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514385+0000", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054201900000000003 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514423+0000", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8306000000000002e-05 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514450+0000", 2024-09-12T09:29:36.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.764e-05 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.029280345, 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010114253, 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 76 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:36.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504665+0000", 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504675+0000", 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0155e-05 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504686+0000", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1034e-05 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504695+0000", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.772e-06 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513322+0000", 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086273440000000003 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513382+0000", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0507000000000002e-05 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513483+0000", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010040299999999999 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514232+0000", 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074900599999999998 2024-09-12T09:29:36.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514714+0000", 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048209099999999998 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514753+0000", 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9110999999999999e-05 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514778+0000", 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5068000000000001e-05 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:90 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.670941056, 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010637117, 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 90 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863005+0000", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7129999999999999e-06 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863170+0000", 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016529899999999999 2024-09-12T09:29:36.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863194+0000", 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4307e-05 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863212+0000", 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7163000000000002e-05 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.865786+0000", 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025744240000000001 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.865911+0000", 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012526200000000001 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.866106+0000", 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019473900000000001 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869618+0000", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035115490000000001 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873348+0000", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037307030000000001 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873522+0000", 2024-09-12T09:29:36.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017342900000000001 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873640+0000", 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118529 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.670694056999999, 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010820224999999999, 2024-09-12T09:29:36.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 91 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863252+0000", 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5659999999999999e-06 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863262+0000", 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-06 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863285+0000", 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3156999999999999e-05 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863305+0000", 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0656e-05 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868464+0000", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051589649999999997 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868586+0000", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121766 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868800+0000", 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021344499999999999 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873803+0000", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035089999999997 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873936+0000", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013331199999999999 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874014+0000", 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7906999999999998e-05 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874070+0000", 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6072000000000001e-05 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.670581532, 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010847973, 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 92 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863364+0000", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863375+0000", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1274000000000001e-05 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863399+0000", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3243000000000001e-05 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863414+0000", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5653000000000001e-05 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869692+0000", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006277721 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869816+0000", 2024-09-12T09:29:36.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012430300000000001 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870084+0000", 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026783200000000002 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873917+0000", 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038328450000000001 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874082+0000", 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557200000000001 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874142+0000", 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.931e-05 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874211+0000", 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8869999999999996e-05 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.670498521999999, 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012610355, 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 93 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863447+0000", 2024-09-12T09:29:36.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4270000000000001e-06 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863601+0000", 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015352900000000001 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863623+0000", 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2506000000000001e-05 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863639+0000", 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6347000000000001e-05 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870855+0000", 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072158680000000003 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870984+0000", 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892599999999999 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.871189+0000", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020435200000000001 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874344+0000", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031557970000000001 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875801+0000", 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001456684 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875949+0000", 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014783600000000001 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876056+0000", 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107083 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:36.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.670270136999999, 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014766520999999999, 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 94 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863676+0000", 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.64e-06 2024-09-12T09:29:36.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863686+0000", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0757000000000001e-05 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863706+0000", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9343000000000001e-05 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863721+0000", 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4946999999999999e-05 2024-09-12T09:29:36.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874405+0000", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010684129000000001 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874528+0000", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012332599999999999 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874725+0000", 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019663600000000001 2024-09-12T09:29:36.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876165+0000", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001440621 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878296+0000", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021301430000000001 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878380+0000", 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4462999999999994e-05 2024-09-12T09:29:36.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878441+0000", 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0516000000000001e-05 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:36.657 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:36.659 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:36.658+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:36.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:29:36.760 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:36.758+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:36.799 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:36.799 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:36.799 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:36.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:36.860 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:36.858+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:36.945 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:36.945 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:36.945 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:36.945 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:36.945 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:36.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:36.960 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:36.958+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:37.062 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:37.058+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.141743+0000", 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.944767350999999, 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024856750000000001, 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.141743+0000", 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.141743+0000", 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.141745+0000", 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6899999999999999e-06 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.141984+0000", 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000239418 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.142010+0000", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5386999999999999e-05 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.142023+0000", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3570000000000001e-05 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.142162+0000", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013885199999999999 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.142200+0000", 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8142999999999997e-05 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144162+0000", 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019620990000000001 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144229+0000", 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6515999999999998e-05 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:21 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'11, mlcod=16'8)", 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.143140+0000", 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.943371018000001, 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029745779999999999, 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143140+0000", 2024-09-12T09:29:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143140+0000", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143142+0000", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0640000000000001e-06 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143262+0000", 2024-09-12T09:29:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120449 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143292+0000", 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143309+0000", 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7039999999999999e-05 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143460+0000", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015119499999999999 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.143496+0000", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5314000000000001e-05 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.146022+0000", 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025262570000000001 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.146114+0000", 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.221e-05 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:22 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'12, mlcod=16'8)", 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.144133+0000", 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.942377616000002, 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020294620000000001, 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144133+0000", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144133+0000", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144135+0000", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.64e-06 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144316+0000", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000180853 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144347+0000", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1223000000000002e-05 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144367+0000", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0689000000000001e-05 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144472+0000", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010432 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144564+0000", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2219000000000006e-05 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.146123+0000", 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001558832 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.146163+0000", 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9685999999999999e-05 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:23 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'13, mlcod=16'8)", 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.144966+0000", 2024-09-12T09:29:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.941545061999999, 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026068250000000001, 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144966+0000", 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144966+0000", 2024-09-12T09:29:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.144967+0000", 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5939999999999999e-06 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.145058+0000", 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0395000000000004e-05 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.145078+0000", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0134000000000001e-05 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.145091+0000", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.145289+0000", 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019764500000000001 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.145320+0000", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.147483+0000", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00216336 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.147572+0000", 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9437000000000004e-05 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:26 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'16, mlcod=16'14)", 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.151632+0000", 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.934878585, 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001704054, 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151632+0000", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151632+0000", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151634+0000", 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5629999999999999e-06 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151643+0000", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6369999999999994e-06 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151669+0000", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5507000000000002e-05 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151703+0000", 2024-09-12T09:29:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4462999999999999e-05 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151838+0000", 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013487199999999999 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.151877+0000", 2024-09-12T09:29:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9016e-05 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.153253+0000", 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013755130000000001 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.153336+0000", 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3483e-05 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:34 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'20, mlcod=16'18)", 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.235342+0000", 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.851168309999998, 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018408599999999999, 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235342+0000", 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235342+0000", 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235347+0000", 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2400000000000001e-06 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235369+0000", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1977e-05 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235394+0000", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5769999999999999e-05 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235411+0000", 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6308999999999999e-05 2024-09-12T09:29:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235573+0000", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016195200000000001 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.235619+0000", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6749999999999998e-05 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.237069+0000", 2024-09-12T09:29:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001449386 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.237183+0000", 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011447600000000001 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:39 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'19)", 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.242253+0000", 2024-09-12T09:29:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.844257251999998, 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024892569999999999, 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242253+0000", 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242253+0000", 2024-09-12T09:29:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242255+0000", 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242283+0000", 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8039999999999999e-05 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242312+0000", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8439e-05 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242328+0000", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6467e-05 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242472+0000", 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014351899999999999 2024-09-12T09:29:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.242515+0000", 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3662999999999999e-05 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244625+0000", 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021098309999999999 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244743+0000", 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117398 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:40 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'19)", 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.842762346000001, 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022420529999999999, 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243751+0000", 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1900000000000002e-06 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243772+0000", 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1965999999999999e-05 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243811+0000", 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8105999999999997e-05 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.243836+0000", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5253999999999999e-05 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244030+0000", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194408 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244068+0000", 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7446e-05 2024-09-12T09:29:37.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.245888+0000", 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018203539999999999 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.245990+0000", 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102329 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:41 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'23, mlcod=16'19)", 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.244818+0000", 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.84169258, 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022903089999999999, 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244818+0000", 2024-09-12T09:29:37.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244818+0000", 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244819+0000", 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9599999999999998e-07 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244904+0000", 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4585000000000004e-05 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244936+0000", 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2780000000000001e-05 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.244949+0000", 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3006999999999999e-05 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.245012+0000", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2157999999999997e-05 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.245031+0000", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9083999999999999e-05 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.247046+0000", 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020156559999999998 2024-09-12T09:29:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.247108+0000", 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2143000000000005e-05 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:44 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'23)", 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.249722+0000", 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.836788992999999, 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021282889999999998, 2024-09-12T09:29:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249722+0000", 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249722+0000", 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249723+0000", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.079e-06 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249787+0000", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3789999999999997e-05 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249802+0000", 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5472e-05 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249812+0000", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5289999999999997e-06 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249923+0000", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000111263 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.249962+0000", 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9233999999999997e-05 2024-09-12T09:29:37.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.251780+0000", 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001818002 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.251850+0000", 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9919999999999995e-05 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:47 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'29, mlcod=16'24)", 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.254161+0000", 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.832349352000001, 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016596930000000001, 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254161+0000", 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254161+0000", 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254163+0000", 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.218e-06 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254224+0000", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1608999999999999e-05 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254242+0000", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7363e-05 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254249+0000", 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3599999999999998e-06 2024-09-12T09:29:37.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254306+0000", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6659000000000001e-05 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.254322+0000", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6007000000000001e-05 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.255776+0000", 2024-09-12T09:29:37.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001454863 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.255821+0000", 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4613999999999997e-05 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:50 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'28)", 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.259957+0000", 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.826553378, 2024-09-12T09:29:37.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001676535, 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.259957+0000", 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.259957+0000", 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.259959+0000", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2470000000000001e-06 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.260043+0000", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4899000000000001e-05 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.260067+0000", 2024-09-12T09:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3356000000000001e-05 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.260082+0000", 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5197e-05 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.260207+0000", 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012478200000000001 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.260245+0000", 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8306999999999998e-05 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.261588+0000", 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001342506 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.261634+0000", 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6241e-05 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:53 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.266001+0000", 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.820509527999999, 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002387583, 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266001+0000", 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266001+0000", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266002+0000", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2130000000000001e-06 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266064+0000", 2024-09-12T09:29:37.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1459999999999998e-05 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266081+0000", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7428999999999999e-05 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266089+0000", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.34e-06 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266185+0000", 2024-09-12T09:29:37.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6521000000000006e-05 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.266221+0000", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5743e-05 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.268300+0000", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020791939999999999 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.268389+0000", 2024-09-12T09:29:37.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8683000000000005e-05 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:54 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'36, mlcod=16'33)", 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.267265+0000", 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.819245713000001, 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019763369999999999, 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267265+0000", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267265+0000", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267266+0000", 2024-09-12T09:29:37.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.33e-06 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267355+0000", 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8665000000000006e-05 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267402+0000", 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6618e-05 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267414+0000", 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2130000000000001e-05 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267519+0000", 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105548 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.267558+0000", 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.837e-05 2024-09-12T09:29:37.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269205+0000", 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016477519999999999 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269241+0000", 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5923999999999997e-05 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:56 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'38, mlcod=16'33)", 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.269694+0000", 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.816817063999999, 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001673405, 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269694+0000", 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269694+0000", 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269694+0000", 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7000000000000003e-07 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269787+0000", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2551000000000002e-05 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269814+0000", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6679e-05 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269823+0000", 2024-09-12T09:29:37.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269955+0000", 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013182299999999999 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.269993+0000", 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7786e-05 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.271335+0000", 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013422239999999999 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.271367+0000", 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2163999999999999e-05 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'47, mlcod=16'42)", 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.506575+0000", 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.579935729999999, 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00177235, 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506575+0000", 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506575+0000", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506576+0000", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5199999999999995e-07 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506650+0000", 2024-09-12T09:29:37.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4091999999999995e-05 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506664+0000", 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3720000000000001e-05 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506672+0000", 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4169999999999992e-06 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506733+0000", 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1264000000000002e-05 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.506749+0000", 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5758000000000001e-05 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.508257+0000", 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015081770000000001 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.508347+0000", 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0069999999999997e-05 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:95 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'69, mlcod=16'67)", 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.878993+0000", 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.207517305, 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001867084, 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.162 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.878993+0000", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.878993+0000", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.878995+0000", 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2130000000000001e-06 2024-09-12T09:29:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.879036+0000", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1779000000000001e-05 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.879054+0000", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7884999999999999e-05 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.879062+0000", 2024-09-12T09:29:37.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2090000000000005e-06 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.879182+0000", 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119102 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.879221+0000", 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9471999999999997e-05 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.880828+0000", 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001606618 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.880860+0000", 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2805999999999997e-05 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4195.0:97 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'69)", 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:17.882909+0000", 2024-09-12T09:29:37.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.203601462000002, 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016231360000000001, 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882909+0000", 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882909+0000", 2024-09-12T09:29:37.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882911+0000", 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8500000000000001e-06 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882923+0000", 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688999999999999e-05 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882952+0000", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9070000000000001e-05 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.882965+0000", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2989999999999999e-05 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.883105+0000", 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014012900000000001 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.883172+0000", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6805999999999997e-05 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.884474+0000", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013023939999999999 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:17.884532+0000", 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8208000000000002e-05 2024-09-12T09:29:37.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.39796261700000002, 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064694160000000004, 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:37.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688551+0000", 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.959e-06 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688564+0000", 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2822e-05 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688589+0000", 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4876999999999999e-05 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688601+0000", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.181e-05 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688688+0000", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7119000000000004e-05 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688696+0000", 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3930000000000008e-06 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.691882+0000", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031855379999999999 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.692106+0000", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022441 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.692449+0000", 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034254799999999997 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.693624+0000", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011752449999999999 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.694902+0000", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012784179999999999 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.694951+0000", 2024-09-12T09:29:37.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8869000000000003e-05 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.695017+0000", 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6408000000000005e-05 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:13 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:36.698067+0000", 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.38844414700000002, 2024-09-12T09:29:37.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025445469999999999, 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698067+0000", 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698067+0000", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698068+0000", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8220000000000001e-06 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698076+0000", 2024-09-12T09:29:37.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0309999999999998e-06 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698095+0000", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8119000000000001e-05 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698112+0000", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7615e-05 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698191+0000", 2024-09-12T09:29:37.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8537999999999999e-05 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698327+0000", 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013679999999999999 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.698487+0000", 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000159498 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:37.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.699283+0000", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00079596199999999999 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.700530+0000", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00124716 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.700572+0000", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2150999999999997e-05 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:37.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.700611+0000", 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8850999999999998e-05 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:37.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:37.185 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:37.182+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:37.286 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:37.286+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:37.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:37.348 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:37.349 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:37.349 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:37.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:37.407 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:37.406+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:37.428 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:37.448 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:37.464 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:37.477 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:37.477 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:37.482 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:37.482 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:37.487 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:37.486+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:37.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:37.503 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:37.503 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:37.523 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:37.523 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:37.544 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:37.545 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:37.567 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:37.588 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:37.586+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:37.614 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:37.614 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:37.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:37.715 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:37.714+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:37.783 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:37.784 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:37.790 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:37.786+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:37.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:37.855 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:29:37.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T09:29:37.927 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:37.926+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:37.954 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:37.954 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:37.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:37.992 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:37.990+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:38.093 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:38.090+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:38.123 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.123 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:38.194 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:38.194+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:38.294 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.294 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.295 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:38.290+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:38.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:38.395 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:38.394+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:38.463 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.463 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:38.496 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:38.494+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:38.597 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:38.594+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:38.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping zero size read 2024-09-12T09:29:38.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping zero length punch hole 2024-09-12T09:29:38.617 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.617 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.630 INFO:teuthology.orchestra.run.smithi007.stdout:5 write 0x3fcccb0 thru 0x3fd8394 (0xb6e5 bytes) 2024-09-12T09:29:38.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:38.637 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.637 INFO:teuthology.orchestra.run.smithi007.stdout:8 read 0x35f6f0d thru 0x35ffe20 (0x8f14 bytes) 2024-09-12T09:29:38.637 INFO:teuthology.orchestra.run.smithi007.stdout:9 read 0x12a0d2 thru 0x1340ea (0xa019 bytes) 2024-09-12T09:29:38.637 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0x1a690a9 2024-09-12T09:29:38.640 INFO:teuthology.orchestra.run.smithi007.stdout:11 trunc from 0x3fd8395 to 0x1a690a9 2024-09-12T09:29:38.656 INFO:teuthology.orchestra.run.smithi007.stdout:12 read 0x1553a5d thru 0x15581bb (0x475f bytes) 2024-09-12T09:29:38.656 INFO:teuthology.orchestra.run.smithi007.stdout:13 read 0x17a0295 thru 0x17ae219 (0xdf85 bytes) 2024-09-12T09:29:38.673 INFO:teuthology.orchestra.run.smithi007.stdout:15 write 0x8a4f512 thru 0x8a4ff72 (0xa61 bytes) 2024-09-12T09:29:38.678 INFO:teuthology.orchestra.run.smithi007.stdout:18 write 0x814419b thru 0x8148861 (0x46c7 bytes) 2024-09-12T09:29:38.691 INFO:teuthology.orchestra.run.smithi007.stdout:19 trunc from 0x8a4ff73 to 0x134d58d 2024-09-12T09:29:38.698 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:38.694+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:38.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:29:38.708 INFO:teuthology.orchestra.run.smithi007.stdout:20 read 0x4cfa5 thru 0x54fe7 (0x8043 bytes) 2024-09-12T09:29:38.708 INFO:teuthology.orchestra.run.smithi007.stdout:21 read 0x739b05 thru 0x73effc (0x54f8 bytes) 2024-09-12T09:29:38.708 INFO:teuthology.orchestra.run.smithi007.stdout:22 read 0x6c64c5 thru 0x6cfacb (0x9607 bytes) 2024-09-12T09:29:38.708 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0x2eee06e 2024-09-12T09:29:38.710 INFO:teuthology.orchestra.run.smithi007.stdout:23 trunc from 0x134d58d to 0x2eee06e 2024-09-12T09:29:38.719 INFO:teuthology.orchestra.run.smithi007.stdout:24 read 0x2832199 thru 0x2835e6f (0x3cd7 bytes) 2024-09-12T09:29:38.719 INFO:teuthology.orchestra.run.smithi007.stdout:25 punch from 0x1785e4 to 0x180d4a, (0x8766 bytes) 2024-09-12T09:29:38.741 INFO:teuthology.orchestra.run.smithi007.stdout:28 writesame 0xec46df6 thru 0xec4ac22 data_size 0x1(0x3e2d bytes) 2024-09-12T09:29:38.741 INFO:teuthology.orchestra.run.smithi007.stdout:29 read 0xbb8b9da thru 0xbb8e726 (0x2d4d bytes) 2024-09-12T09:29:38.741 INFO:teuthology.orchestra.run.smithi007.stdout:30 write 0xa3d39cb thru 0xa3e3465 (0xfa9b bytes) 2024-09-12T09:29:38.749 INFO:teuthology.orchestra.run.smithi007.stdout:32 read 0x5dd38d9 thru 0x5dda067 (0x678f bytes) 2024-09-12T09:29:38.749 INFO:teuthology.orchestra.run.smithi007.stdout:33 punch from 0x9563c58 to 0x956e707, (0xaaaf bytes) 2024-09-12T09:29:38.749 INFO:teuthology.orchestra.run.smithi007.stdout:34 write 0x9555022 thru 0x9557589 (0x2568 bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:36 punch from 0x117a506 to 0x1189215, (0xed0f bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:37 read 0x921e184 thru 0x92210e8 (0x2f65 bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:38 punch from 0x6697d0 to 0x673124, (0x9954 bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:40 punch from 0x8583620 to 0x85847ea, (0x11ca bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:42 read 0xe33c9c0 thru 0xe33f5e9 (0x2c2a bytes) 2024-09-12T09:29:38.755 INFO:teuthology.orchestra.run.smithi007.stdout:43 writesame 0x74304c5 thru 0x74314ba data_size 0x9(0xff6 bytes) 2024-09-12T09:29:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:45 punch from 0xa5a506c to 0xa5a7d2c, (0x2cc0 bytes) 2024-09-12T09:29:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:47 write 0x8069cbe thru 0x806c457 (0x279a bytes) 2024-09-12T09:29:38.764 INFO:teuthology.orchestra.run.smithi007.stdout:48 read 0x55a8a53 thru 0x55a9741 (0xcef bytes) 2024-09-12T09:29:38.765 INFO:teuthology.orchestra.run.smithi007.stdout:49 write 0xa465dc6 thru 0xa46b76d (0x59a8 bytes) 2024-09-12T09:29:38.769 INFO:teuthology.orchestra.run.smithi007.stdout:51 write 0x6b7d29f thru 0x6b88cd2 (0xba34 bytes) 2024-09-12T09:29:38.770 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.770 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:53 read 0x5f4d03d thru 0x5f56d55 (0x9d19 bytes) 2024-09-12T09:29:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0x6963449 2024-09-12T09:29:38.784 INFO:teuthology.orchestra.run.smithi007.stdout:54 trunc from 0xec4ac23 to 0x6963449 2024-09-12T09:29:38.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:38.799 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.799 INFO:teuthology.orchestra.run.smithi007.stdout:56 read 0x4819fe4 thru 0x4824df3 (0xae10 bytes) 2024-09-12T09:29:38.800 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0x9da1b0f 2024-09-12T09:29:38.803 INFO:teuthology.orchestra.run.smithi007.stdout:57 trunc from 0x6963449 to 0x9da1b0f 2024-09-12T09:29:38.811 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.823 INFO:teuthology.orchestra.run.smithi007.stdout:59 write 0xb75776c thru 0xb75f8d9 (0x816e bytes) 2024-09-12T09:29:38.828 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:38.831 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:38.830+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:38.837 INFO:teuthology.orchestra.run.smithi007.stdout:61 writesame 0xc5a9030 thru 0xc5a9c68 data_size 0x7(0xc39 bytes) 2024-09-12T09:29:38.846 INFO:teuthology.orchestra.run.smithi007.stdout:62 write 0xdf726b1 thru 0xdf73d74 (0x16c4 bytes) 2024-09-12T09:29:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:63 punch from 0x457547b to 0x457a126, (0x4cab bytes) 2024-09-12T09:29:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:64 read 0x5df4ac3 thru 0x5dfd1ac (0x86ea bytes) 2024-09-12T09:29:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:65 read 0x9ace137 thru 0x9ad92d1 (0xb19b bytes) 2024-09-12T09:29:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:66 trunc from 0xdf73d75 to 0x2420641 2024-09-12T09:29:38.889 INFO:teuthology.orchestra.run.smithi007.stdout:67 write 0x72881b2 thru 0x728aec5 (0x2d14 bytes) 2024-09-12T09:29:38.899 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:38.898+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:38.910 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:38.910 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:38.912 INFO:teuthology.orchestra.run.smithi007.stdout:68 write 0xee39654 thru 0xee48bfd (0xf5aa bytes) 2024-09-12T09:29:38.918 INFO:teuthology.orchestra.run.smithi007.stdout:70 read 0xa773068 thru 0xa780cf5 (0xdc8e bytes) 2024-09-12T09:29:38.918 INFO:teuthology.orchestra.run.smithi007.stdout:71 write 0x6ae5227 thru 0x6aee708 (0x94e2 bytes) 2024-09-12T09:29:38.922 INFO:teuthology.orchestra.run.smithi007.stdout:73 writesame 0x1839b4c thru 0x183d4dc data_size 0x1(0x3991 bytes) 2024-09-12T09:29:38.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:38.937 INFO:teuthology.orchestra.run.smithi007.stdout:75 punch from 0x6dd33c2 to 0x6dd46f7, (0x1335 bytes) 2024-09-12T09:29:38.937 INFO:teuthology.orchestra.run.smithi007.stdout:76 write 0xc16e785 thru 0xc17057b (0x1df7 bytes) 2024-09-12T09:29:38.943 INFO:teuthology.orchestra.run.smithi007.stdout:77 writesame 0xc1f2a38 thru 0xc1f45f1 data_size 0x1a(0x1bba bytes) 2024-09-12T09:29:38.945 INFO:teuthology.orchestra.run.smithi007.stdout:78 read 0x5a04758 thru 0x5a0f4b9 (0xad62 bytes) 2024-09-12T09:29:38.945 INFO:teuthology.orchestra.run.smithi007.stdout:80 write 0x376d0de thru 0x3773b1d (0x6a40 bytes) 2024-09-12T09:29:38.950 INFO:teuthology.orchestra.run.smithi007.stdout:81 read 0xb8c22ec thru 0xb8d07b7 (0xe4cc bytes) 2024-09-12T09:29:38.966 INFO:teuthology.orchestra.run.smithi007.stdout:82 trunc from 0xee48bfe to 0x5e983bb 2024-09-12T09:29:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T09:29:39.001 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:38.998+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:39.018 INFO:teuthology.orchestra.run.smithi007.stdout:83 writesame 0xd7c899a thru 0xd7d81c6 data_size 0x1(0xf82d bytes) 2024-09-12T09:29:39.018 INFO:teuthology.orchestra.run.smithi007.stdout:85 read 0x10779b6 thru 0x1086eea (0xf535 bytes) 2024-09-12T09:29:39.018 INFO:teuthology.orchestra.run.smithi007.stdout:86 read 0x7d8d43a thru 0x7d8df84 (0xb4b bytes) 2024-09-12T09:29:39.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T09:29:39.022 INFO:teuthology.orchestra.run.smithi007.stdout:87 trunc from 0xd7d81c7 to 0x9c970ab 2024-09-12T09:29:39.029 INFO:teuthology.orchestra.run.smithi007.stdout:88 write 0x8282864 thru 0x82872c5 (0x4a62 bytes) 2024-09-12T09:29:39.033 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.033 INFO:teuthology.orchestra.run.smithi007.stdout:90 write 0x3075929 thru 0x30819dd (0xc0b5 bytes) 2024-09-12T09:29:39.048 INFO:teuthology.orchestra.run.smithi007.stdout:91 write 0xcd23edb thru 0xcd2d61a (0x9740 bytes) 2024-09-12T09:29:39.058 INFO:teuthology.orchestra.run.smithi007.stdout:92 trunc from 0xcd2d61b to 0x842c125 2024-09-12T09:29:39.083 INFO:teuthology.orchestra.run.smithi007.stdout:94 writesame 0xdff16fc thru 0xdff8541 data_size 0xf(0x6e46 bytes) 2024-09-12T09:29:39.083 INFO:teuthology.orchestra.run.smithi007.stdout:95 write 0x287b1fe thru 0x287e443 (0x3246 bytes) 2024-09-12T09:29:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:96 read 0x2cc1290 thru 0x2cc3846 (0x25b7 bytes) 2024-09-12T09:29:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:97 write 0xb23f915 thru 0xb24d3a3 (0xda8f bytes) 2024-09-12T09:29:39.099 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.099 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.100 INFO:teuthology.orchestra.run.smithi007.stdout:99 writesame 0xe68f8e9 thru 0xe698dd8 data_size 0x8(0x94f0 bytes) 2024-09-12T09:29:39.101 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.101 INFO:teuthology.orchestra.run.smithi007.stdout:101 writesame 0xd2a9ecf thru 0xd2ac328 data_size 0xb(0x245a bytes) 2024-09-12T09:29:39.101 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:39.098+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:39.104 INFO:teuthology.orchestra.run.smithi007.stdout:102 read 0xc3f782c thru 0xc4010d7 (0x98ac bytes) 2024-09-12T09:29:39.104 INFO:teuthology.orchestra.run.smithi007.stdout:104 write 0x2b13d8b thru 0x2b1d3c0 (0x9636 bytes) 2024-09-12T09:29:39.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:39.120 INFO:teuthology.orchestra.run.smithi007.stdout:105 trunc from 0xe698dd9 to 0x44e92df 2024-09-12T09:29:39.133 INFO:teuthology.orchestra.run.smithi007.stdout:107 read 0x3ede645 thru 0x3ee7651 (0x900d bytes) 2024-09-12T09:29:39.133 INFO:teuthology.orchestra.run.smithi007.stdout:108 read 0x22a5a90 thru 0x22b296c (0xcedd bytes) 2024-09-12T09:29:39.134 INFO:teuthology.orchestra.run.smithi007.stdout:109 read 0x262b188 thru 0x263a990 (0xf809 bytes) 2024-09-12T09:29:39.134 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.134 INFO:teuthology.orchestra.run.smithi007.stdout:111 read 0x5b17e9 thru 0x5b245b (0xc73 bytes) 2024-09-12T09:29:39.141 INFO:teuthology.orchestra.run.smithi007.stdout:112 write 0x5ed90e5 thru 0x5ede16b (0x5087 bytes) 2024-09-12T09:29:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:113 trunc from 0x5ede16c to 0x7cfc186 2024-09-12T09:29:39.153 INFO:teuthology.orchestra.run.smithi007.stdout:114 read 0x1cacf4 thru 0x1d9465 (0xe772 bytes) 2024-09-12T09:29:39.163 INFO:teuthology.orchestra.run.smithi007.stdout:115 trunc from 0x7cfc186 to 0x2fdb3a 2024-09-12T09:29:39.177 INFO:teuthology.orchestra.run.smithi007.stdout:116 punch from 0xa4359 to 0xac90e, (0x85b5 bytes) 2024-09-12T09:29:39.197 INFO:teuthology.orchestra.run.smithi007.stdout:119 write 0x8669988 thru 0x8671f6d (0x85e6 bytes) 2024-09-12T09:29:39.202 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:39.198+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:39.210 INFO:teuthology.orchestra.run.smithi007.stdout:120 trunc from 0x8671f6e to 0x3c8dc4e 2024-09-12T09:29:39.223 INFO:teuthology.orchestra.run.smithi007.stdout:121 write 0x285aae thru 0x287b8a (0x20dd bytes) 2024-09-12T09:29:39.232 INFO:teuthology.orchestra.run.smithi007.stdout:122 trunc from 0x3c8dc4e to 0xd34564 2024-09-12T09:29:39.233 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.234 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.248 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:39.255 INFO:teuthology.orchestra.run.smithi007.stdout:123 write 0x5235688 thru 0x52392da (0x3c53 bytes) 2024-09-12T09:29:39.261 INFO:teuthology.orchestra.run.smithi007.stdout:124 trunc from 0x52392db to 0x4b8d3cf 2024-09-12T09:29:39.277 INFO:teuthology.orchestra.run.smithi007.stdout:125 read 0x2a4cf57 thru 0x2a57a4d (0xaaf7 bytes) 2024-09-12T09:29:39.277 INFO:teuthology.orchestra.run.smithi007.stdout:126 read 0x1cd4d08 thru 0x1cd4ebe (0x1b7 bytes) 2024-09-12T09:29:39.280 INFO:teuthology.orchestra.run.smithi007.stdout:127 trunc from 0x4b8d3cf to 0x2beff02 2024-09-12T09:29:39.290 INFO:teuthology.orchestra.run.smithi007.stdout:128 writesame 0xf4bb2c thru 0xf515ec data_size 0x1(0x5ac1 bytes) 2024-09-12T09:29:39.303 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:39.298+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:39.307 INFO:teuthology.orchestra.run.smithi007.stdout:129 read 0x232dd0a thru 0x233606c (0x8363 bytes) 2024-09-12T09:29:39.307 INFO:teuthology.orchestra.run.smithi007.stdout:130 punch from 0x11d131a to 0x11d1cc7, (0x9ad bytes) 2024-09-12T09:29:39.307 INFO:teuthology.orchestra.run.smithi007.stdout:131 read 0x259d239 thru 0x259da46 (0x80e bytes) 2024-09-12T09:29:39.307 INFO:teuthology.orchestra.run.smithi007.stdout:132 punch from 0x64e6dc to 0x65d838, (0xf15c bytes) 2024-09-12T09:29:39.320 INFO:teuthology.orchestra.run.smithi007.stdout:133 write 0x3b621b4 thru 0x3b71b6a (0xf9b7 bytes) 2024-09-12T09:29:39.325 INFO:teuthology.orchestra.run.smithi007.stdout:135 read 0x249cc41 thru 0x24a248c (0x584c bytes) 2024-09-12T09:29:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:136 writesame 0x89ec57c thru 0x89f8700 data_size 0x1(0xc185 bytes) 2024-09-12T09:29:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:139 writesame 0x501867d thru 0x501cfb5 data_size 0x17(0x4939 bytes) 2024-09-12T09:29:39.352 INFO:teuthology.orchestra.run.smithi007.stdout:141 punch from 0x42e0d20 to 0x42f026f, (0xf54f bytes) 2024-09-12T09:29:39.367 INFO:teuthology.orchestra.run.smithi007.stdout:143 writesame 0x9fdb356 thru 0x9fdf697 data_size 0x1(0x4342 bytes) 2024-09-12T09:29:39.367 INFO:teuthology.orchestra.run.smithi007.stdout:145 read 0x795ab31 thru 0x796717e (0xc64e bytes) 2024-09-12T09:29:39.367 INFO:teuthology.orchestra.run.smithi007.stdout:146 read 0x5856d1b thru 0x585921b (0x2501 bytes) 2024-09-12T09:29:39.367 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.368 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.368 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.378 INFO:teuthology.orchestra.run.smithi007.stdout:148 trunc from 0x9fdf698 to 0x45880ca 2024-09-12T09:29:39.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:39.397 INFO:teuthology.orchestra.run.smithi007.stdout:150 read 0x139eaab thru 0x13a25da (0x3b30 bytes) 2024-09-12T09:29:39.402 INFO:teuthology.orchestra.run.smithi007.stdout:151 trunc from 0x45880ca to 0x1e11d84 2024-09-12T09:29:39.403 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:39.398+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:39.417 INFO:teuthology.orchestra.run.smithi007.stdout:152 read 0xa2c18c thru 0xa349d7 (0x884c bytes) 2024-09-12T09:29:39.418 INFO:teuthology.orchestra.run.smithi007.stdout:153 read 0x44e092 thru 0x454b48 (0x6ab7 bytes) 2024-09-12T09:29:39.418 INFO:teuthology.orchestra.run.smithi007.stdout:154 punch from 0x9186ca to 0x925a62, (0xd398 bytes) 2024-09-12T09:29:39.444 INFO:teuthology.orchestra.run.smithi007.stdout:155 write 0xb0ac72b thru 0xb0bc657 (0xff2d bytes) 2024-09-12T09:29:39.450 INFO:teuthology.orchestra.run.smithi007.stdout:156 write 0xd4e8f7 thru 0xd5cfa6 (0xe6b0 bytes) 2024-09-12T09:29:39.454 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.455 INFO:teuthology.orchestra.run.smithi007.stdout:158 punch from 0x2c7b883 to 0x2c7f80e, (0x3f8b bytes) 2024-09-12T09:29:39.470 INFO:teuthology.orchestra.run.smithi007.stdout:159 writesame 0xce5b005 thru 0xce61112 data_size 0x6(0x610e bytes) 2024-09-12T09:29:39.479 INFO:teuthology.orchestra.run.smithi007.stdout:160 trunc from 0xce61113 to 0x8aac6ba 2024-09-12T09:29:39.501 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.502 INFO:teuthology.orchestra.run.smithi007.stdout:163 punch from 0x5abf754 to 0x5ac485a, (0x5106 bytes) 2024-09-12T09:29:39.502 INFO:teuthology.orchestra.run.smithi007.stdout:164 read 0x6a4544a thru 0x6a4e210 (0x8dc7 bytes) 2024-09-12T09:29:39.502 INFO:teuthology.orchestra.run.smithi007.stdout:165 read 0xa32d47 thru 0xa3b999 (0x8c53 bytes) 2024-09-12T09:29:39.502 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.503 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:39.502+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:39.506 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.507 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:39.523 INFO:teuthology.orchestra.run.smithi007.stdout:167 trunc from 0x8aac6ba to 0x76a251 2024-09-12T09:29:39.574 INFO:teuthology.orchestra.run.smithi007.stdout:168 write 0x6f3bd71 thru 0x6f472d9 (0xb569 bytes) 2024-09-12T09:29:39.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:39.594 INFO:teuthology.orchestra.run.smithi007.stdout:170 trunc from 0x6f472da to 0x240a294 2024-09-12T09:29:39.604 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:39.602+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:172 writesame 0x522b834 thru 0x5234487 data_size 0xe(0x8c54 bytes) 2024-09-12T09:29:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:173 read 0x50d58f1 thru 0x50de1d1 (0x88e1 bytes) 2024-09-12T09:29:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:177 read 0x3b87b2c thru 0x3b8b7e8 (0x3cbd bytes) 2024-09-12T09:29:39.638 INFO:teuthology.orchestra.run.smithi007.stdout:178 writesame 0x6910b5d thru 0x69111dd data_size 0xf(0x681 bytes) 2024-09-12T09:29:39.639 INFO:teuthology.orchestra.run.smithi007.stdout:179 punch from 0x2e981a1 to 0x2e9b8f4, (0x3753 bytes) 2024-09-12T09:29:39.649 INFO:teuthology.orchestra.run.smithi007.stdout:180 writesame 0x7e31438 thru 0x7e39e4e data_size 0x1d(0x8a17 bytes) 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:181 writesame 0xe42a846 thru 0xe42b6bb data_size 0x3(0xe76 bytes) 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:182 read 0x7dcf666 thru 0x7dd815d (0x8af8 bytes) 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.670 INFO:teuthology.orchestra.run.smithi007.stdout:187 write 0x8847a05 thru 0x884c61f (0x4c1b bytes) 2024-09-12T09:29:39.677 INFO:teuthology.orchestra.run.smithi007.stdout:188 read 0xdf3b465 thru 0xdf47d8e (0xc92a bytes) 2024-09-12T09:29:39.677 INFO:teuthology.orchestra.run.smithi007.stdout:189 write 0xb6f2742 thru 0xb6ff2bb (0xcb7a bytes) 2024-09-12T09:29:39.684 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.684 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.700 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:39.701 INFO:teuthology.orchestra.run.smithi007.stdout:190 write 0xe951d3c thru 0xe95da90 (0xbd55 bytes) 2024-09-12T09:29:39.705 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:39.702+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:39.706 INFO:teuthology.orchestra.run.smithi007.stdout:191 write 0xe6b6c16 thru 0xe6bff70 (0x935b bytes) 2024-09-12T09:29:39.711 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xac2aa0c 2024-09-12T09:29:39.719 INFO:teuthology.orchestra.run.smithi007.stdout:192 trunc from 0xe95da91 to 0xac2aa0c 2024-09-12T09:29:39.737 INFO:teuthology.orchestra.run.smithi007.stdout:193 write 0x889f142 thru 0x88a9199 (0xa058 bytes) 2024-09-12T09:29:39.739 INFO:teuthology.orchestra.run.smithi007.stdout:194 writesame 0x7c43b55 thru 0x7c4e15e data_size 0x2(0xa60a bytes) 2024-09-12T09:29:39.752 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.752 INFO:teuthology.orchestra.run.smithi007.stdout:196 read 0x40a3651 thru 0x40ae139 (0xaae9 bytes) 2024-09-12T09:29:39.752 INFO:teuthology.orchestra.run.smithi007.stdout:197 writesame 0x173ba27 thru 0x1744d8a data_size 0x2(0x9364 bytes) 2024-09-12T09:29:39.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:39.775 INFO:teuthology.orchestra.run.smithi007.stdout:200 writesame 0xd2f8459 thru 0xd305647 data_size 0x1(0xd1ef bytes) 2024-09-12T09:29:39.775 INFO:teuthology.orchestra.run.smithi007.stdout:202 punch from 0x22b7cfd to 0x22be1f7, (0x64fa bytes) 2024-09-12T09:29:39.775 INFO:teuthology.orchestra.run.smithi007.stdout:203 read 0x1b89d2a thru 0x1b95b3f (0xbe16 bytes) 2024-09-12T09:29:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:205 write 0xb08c472 thru 0xb08cac5 (0x654 bytes) 2024-09-12T09:29:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:206 punch from 0x3b87832 to 0x3b9743a, (0xfc08 bytes) 2024-09-12T09:29:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:208 punch from 0xd210439 to 0xd210503, (0xca bytes) 2024-09-12T09:29:39.780 INFO:teuthology.orchestra.run.smithi007.stdout:210 read 0xa5f2749 thru 0xa5fcfb2 (0xa86a bytes) 2024-09-12T09:29:39.780 INFO:teuthology.orchestra.run.smithi007.stdout:211 write 0x49220c3 thru 0x492b41b (0x9359 bytes) 2024-09-12T09:29:39.782 INFO:teuthology.orchestra.run.smithi007.stdout:212 read 0x7c95bff thru 0x7ca2b6a (0xcf6c bytes) 2024-09-12T09:29:39.783 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xe672e23 2024-09-12T09:29:39.784 INFO:teuthology.orchestra.run.smithi007.stdout:213 trunc from 0xd305648 to 0xe672e23 2024-09-12T09:29:39.787 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:39.788 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:39.790 INFO:teuthology.orchestra.run.smithi007.stdout:214 read 0x391b706 thru 0x392889b (0xd196 bytes) 2024-09-12T09:29:39.790 INFO:teuthology.orchestra.run.smithi007.stdout:215 punch from 0xb1e3ca9 to 0xb1f30ac, (0xf403 bytes) 2024-09-12T09:29:39.790 INFO:teuthology.orchestra.run.smithi007.stdout:217 write 0x55c8ec0 thru 0x55cf9d4 (0x6b15 bytes) 2024-09-12T09:29:39.794 INFO:teuthology.orchestra.run.smithi007.stdout:219 writesame 0xc483739 thru 0xc48740e data_size 0xd(0x3cd6 bytes) 2024-09-12T09:29:39.799 INFO:teuthology.orchestra.run.smithi007.stdout:221 trunc from 0xe672e23 to 0xdbb71a5 2024-09-12T09:29:39.806 INFO:teuthology.orchestra.run.smithi007.stdout:222 read 0x7726e65 thru 0x77275d9 (0x775 bytes) 2024-09-12T09:29:39.806 INFO:teuthology.orchestra.run.smithi007.stdout:223 read 0x61f4bed thru 0x61f88e2 (0x3cf6 bytes) 2024-09-12T09:29:39.806 INFO:teuthology.orchestra.run.smithi007.stdout:224 read 0xab64905 thru 0xab6f6de (0xadda bytes) 2024-09-12T09:29:39.806 INFO:teuthology.orchestra.run.smithi007.stdout:227 writesame 0x9183d21 thru 0x9184f62 data_size 0x13(0x1242 bytes) 2024-09-12T09:29:39.807 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:39.802+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:39.807 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:39.807 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:39.809 INFO:teuthology.orchestra.run.smithi007.stdout:228 write 0x3524ea8 thru 0x3528832 (0x398b bytes) 2024-09-12T09:29:39.813 INFO:teuthology.orchestra.run.smithi007.stdout:230 write 0xc3a5528 thru 0xc3a5674 (0x14d bytes) 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:232 read 0x7ab0a68 thru 0x7ac05fd (0xfb96 bytes) 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:236 punch from 0x6e7fd90 to 0x6e8766d, (0x78dd bytes) 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.819 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:39.819 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:39.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:39.831 INFO:teuthology.orchestra.run.smithi007.stdout:239 trunc from 0xdbb71a5 to 0x5c3008a 2024-09-12T09:29:39.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:240 read 0x7f69ed thru 0x7fb6bb (0x4ccf bytes) 2024-09-12T09:29:39.846 INFO:teuthology.orchestra.run.smithi007.stdout:241 read 0x632d76 thru 0x63ba02 (0x8c8d bytes) 2024-09-12T09:29:39.856 INFO:teuthology.orchestra.run.smithi007.stdout:242 writesame 0x822d800 thru 0x82384bc data_size 0x1(0xacbd bytes) 2024-09-12T09:29:39.856 INFO:teuthology.orchestra.run.smithi007.stdout:244 read 0x193579f thru 0x193cfaa (0x780c bytes) 2024-09-12T09:29:39.856 INFO:teuthology.orchestra.run.smithi007.stdout:245 read 0x526c80 thru 0x52b038 (0x43b9 bytes) 2024-09-12T09:29:39.856 INFO:teuthology.orchestra.run.smithi007.stdout:247 punch from 0x72dc62 to 0x73b569, (0xd907 bytes) 2024-09-12T09:29:39.856 INFO:teuthology.orchestra.run.smithi007.stdout:248 read 0x424b5d3 thru 0x425b3a2 (0xfdd0 bytes) 2024-09-12T09:29:39.857 INFO:teuthology.orchestra.run.smithi007.stdout:249 trunc from 0x82384bd to 0x9f5d297 2024-09-12T09:29:39.866 INFO:teuthology.orchestra.run.smithi007.stdout:252 trunc from 0x9f5d297 to 0xb6453f3 2024-09-12T09:29:39.867 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:39.872 INFO:teuthology.orchestra.run.smithi007.stdout:253 punch from 0x50690fc to 0x506a645, (0x1549 bytes) 2024-09-12T09:29:39.872 INFO:teuthology.orchestra.run.smithi007.stdout:254 write 0x36a5fb1 thru 0x36b4022 (0xe072 bytes) 2024-09-12T09:29:39.874 INFO:teuthology.orchestra.run.smithi007.stdout:256 write 0xaf108cf thru 0xaf1b4ec (0xac1e bytes) 2024-09-12T09:29:39.879 INFO:teuthology.orchestra.run.smithi007.stdout:257 read 0x708f501 thru 0x7099825 (0xa325 bytes) 2024-09-12T09:29:39.879 INFO:teuthology.orchestra.run.smithi007.stdout:259 writesame 0x183462f thru 0x1837a3e data_size 0x31(0x3410 bytes) 2024-09-12T09:29:39.883 INFO:teuthology.orchestra.run.smithi007.stdout:260 writesame 0x2ce56eb thru 0x2cea047 data_size 0x1(0x495d bytes) 2024-09-12T09:29:39.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:39.899 INFO:teuthology.orchestra.run.smithi007.stdout:261 read 0x22f087b thru 0x22f65f9 (0x5d7f bytes) 2024-09-12T09:29:39.899 INFO:teuthology.orchestra.run.smithi007.stdout:263 read 0x17c4d8b thru 0x17c64cf (0x1745 bytes) 2024-09-12T09:29:39.902 INFO:teuthology.orchestra.run.smithi007.stdout:265 trunc from 0xb6453f3 to 0x9013d7a 2024-09-12T09:29:39.902 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:39.902 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:39.914 INFO:teuthology.orchestra.run.smithi007.stdout:268 punch from 0xf86e3 to 0x10119c, (0x8ab9 bytes) 2024-09-12T09:29:39.914 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.914 INFO:teuthology.orchestra.run.smithi007.stdout:270 read 0x7bd8ce6 thru 0x7be8444 (0xf75f bytes) 2024-09-12T09:29:39.914 INFO:teuthology.orchestra.run.smithi007.stdout:271 write 0x7240f72 thru 0x724da78 (0xcb07 bytes) 2024-09-12T09:29:39.918 INFO:teuthology.orchestra.run.smithi007.stdout:272 write 0x2e0c32b thru 0x2e17163 (0xae39 bytes) 2024-09-12T09:29:39.919 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:39.918+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:39.920 INFO:teuthology.orchestra.run.smithi007.stdout:273 trunc from 0x9013d7a to 0x9531d62 2024-09-12T09:29:39.935 INFO:teuthology.orchestra.run.smithi007.stdout:274 write 0xc65822c thru 0xc65de30 (0x5c05 bytes) 2024-09-12T09:29:39.946 INFO:teuthology.orchestra.run.smithi007.stdout:276 trunc from 0xc65de31 to 0x587980b 2024-09-12T09:29:39.971 INFO:teuthology.orchestra.run.smithi007.stdout:277 write 0xcfe069d thru 0xcfeaf41 (0xa8a5 bytes) 2024-09-12T09:29:39.975 INFO:teuthology.orchestra.run.smithi007.stdout:278 read 0x1f53f2a thru 0x1f637d6 (0xf8ad bytes) 2024-09-12T09:29:39.982 INFO:teuthology.orchestra.run.smithi007.stdout:280 trunc from 0xcfeaf42 to 0x73075fd 2024-09-12T09:29:39.995 INFO:teuthology.orchestra.run.smithi007.stdout:282 punch from 0x17d2287 to 0x17da55f, (0x82d8 bytes) 2024-09-12T09:29:39.996 INFO:teuthology.orchestra.run.smithi007.stdout:283 read 0x324e762 thru 0x324f8cd (0x116c bytes) 2024-09-12T09:29:39.996 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:39.996 INFO:teuthology.orchestra.run.smithi007.stdout:285 write 0x40e5caa thru 0x40f3aca (0xde21 bytes) 2024-09-12T09:29:40.002 INFO:teuthology.orchestra.run.smithi007.stdout:286 writesame 0x6aa687c thru 0x6aa837c data_size 0x1(0x1b01 bytes) 2024-09-12T09:29:40.008 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:40.006+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:40.011 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:745 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2363250~51975] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.094656494999999993, 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001823648, 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 745 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:40.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918404+0000", 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0159999999999998e-06 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918462+0000", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.834e-05 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918483+0000", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0208999999999999e-05 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918492+0000", 2024-09-12T09:29:40.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3489999999999998e-06 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918562+0000", 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0344000000000003e-05 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918635+0000", 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2742000000000003e-05 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918761+0000", 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126291 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.919854+0000", 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010925830000000001 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920156+0000", 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249900000000002 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920193+0000", 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6974999999999998e-05 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920226+0000", 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2299999999999999e-05 2024-09-12T09:29:40.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:745 3.a e27/23 ondisk, result = 0)", 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.092988024000000002, 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000159746, 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920072+0000", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920078+0000", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2700000000000001e-06 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920093+0000", 2024-09-12T09:29:40.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5196e-05 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920101+0000", 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920148+0000", 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6177000000000002e-05 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920153+0000", 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.587e-06 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920230+0000", 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7127999999999997e-05 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:768 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.061728218000000001, 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011468400000000001, 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 768 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:40.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951332+0000", 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1009999999999999e-06 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951339+0000", 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1789999999999996e-06 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951355+0000", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6075000000000001e-05 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951367+0000", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2342e-05 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951432+0000", 2024-09-12T09:29:40.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5189000000000002e-05 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951513+0000", 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0994000000000004e-05 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951605+0000", 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1802e-05 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952293+0000", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068794799999999997 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952412+0000", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118979 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952445+0000", 2024-09-12T09:29:40.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.277e-05 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952477+0000", 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2461e-05 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:768 3.a e27/23 ondisk, result = 0)", 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.060715802999999999, 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013823000000000001, 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:40.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:40.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-07 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952348+0000", 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4190000000000004e-06 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952358+0000", 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0297e-05 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952365+0000", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3169999999999998e-06 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952404+0000", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8986000000000002e-05 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952409+0000", 2024-09-12T09:29:40.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.203e-06 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952481+0000", 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2230000000000005e-05 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.036 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:40.037 INFO:teuthology.orchestra.run.smithi007.stdout:287 read 0x27b1ac0 thru 0x27b21da (0x71b bytes) 2024-09-12T09:29:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:29:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T09:29:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:288 write 0xee158fc thru 0xee16727 (0xe2c bytes) 2024-09-12T09:29:40.039 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:40.041 INFO:teuthology.orchestra.run.smithi007.stdout:289 write 0x34add66 thru 0x34b52f0 (0x758b bytes) 2024-09-12T09:29:40.046 INFO:teuthology.orchestra.run.smithi007.stdout:292 writesame 0x90672be thru 0x906e8be data_size 0x1(0x7601 bytes) 2024-09-12T09:29:40.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T09:29:40.065 INFO:teuthology.orchestra.run.smithi007.stdout:293 read 0x25bd88 thru 0x2663e2 (0xa65b bytes) 2024-09-12T09:29:40.066 INFO:teuthology.orchestra.run.smithi007.stdout:294 read 0xd943ff7 thru 0xd94e0c1 (0xa0cb bytes) 2024-09-12T09:29:40.067 INFO:teuthology.orchestra.run.smithi007.stdout:296 writesame 0xe956d33 thru 0xe9571fc data_size 0x1(0x4ca bytes) 2024-09-12T09:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.071 INFO:teuthology.orchestra.run.smithi007.stdout:298 writesame 0x4e403cd thru 0x4e4b3af data_size 0x3(0xafe3 bytes) 2024-09-12T09:29:40.081 INFO:teuthology.orchestra.run.smithi007.stdout:299 write 0x9d9ec67 thru 0x9dab73f (0xcad9 bytes) 2024-09-12T09:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout:300 punch from 0xdd23bed to 0xdd2efb1, (0xb3c4 bytes) 2024-09-12T09:29:40.086 INFO:teuthology.orchestra.run.smithi007.stdout:301 writesame 0xd74dcc7 thru 0xd75c584 data_size 0x1f(0xe8be bytes) 2024-09-12T09:29:40.092 INFO:teuthology.orchestra.run.smithi007.stdout:302 writesame 0x272d0f4 thru 0x2736097 data_size 0x4(0x8fa4 bytes) 2024-09-12T09:29:40.101 INFO:teuthology.orchestra.run.smithi007.stdout:303 trunc from 0xee16728 to 0xe147085 2024-09-12T09:29:40.108 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:40.106+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout:305 read 0x4d0abad thru 0x4d18141 (0xd595 bytes) 2024-09-12T09:29:40.113 INFO:teuthology.orchestra.run.smithi007.stdout:306 write 0xd76f62f thru 0xd76fafe (0x4d0 bytes) 2024-09-12T09:29:40.114 INFO:teuthology.orchestra.run.smithi007.stdout:309 write 0xc39504f thru 0xc3957c7 (0x779 bytes) 2024-09-12T09:29:40.118 INFO:teuthology.orchestra.run.smithi007.stdout:310 writesame 0xbed2fec thru 0xbedf309 data_size 0x96(0xc31e bytes) 2024-09-12T09:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.122 INFO:teuthology.orchestra.run.smithi007.stdout:312 write 0x9993439 thru 0x99a2c1a (0xf7e2 bytes) 2024-09-12T09:29:40.129 INFO:teuthology.orchestra.run.smithi007.stdout:314 write 0x8e72ff9 thru 0x8e7e301 (0xb309 bytes) 2024-09-12T09:29:40.142 INFO:teuthology.orchestra.run.smithi007.stdout:318 trunc from 0xe147085 to 0x62363bb 2024-09-12T09:29:40.170 INFO:teuthology.orchestra.run.smithi007.stdout:319 writesame 0x829dc37 thru 0x82a4df3 data_size 0x1(0x71bd bytes) 2024-09-12T09:29:40.172 INFO:teuthology.orchestra.run.smithi007.stdout:322 trunc from 0x82a4df4 to 0x93b2484 2024-09-12T09:29:40.187 INFO:teuthology.orchestra.run.smithi007.stdout:323 trunc from 0x93b2484 to 0x1c4c91c 2024-09-12T09:29:40.204 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.204 INFO:teuthology.orchestra.run.smithi007.stdout:325 punch from 0x17b35e2 to 0x17bf792, (0xc1b0 bytes) 2024-09-12T09:29:40.209 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:40.206+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:40.217 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:40.217 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:40.217 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:29:40.217 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:40.223 INFO:teuthology.orchestra.run.smithi007.stdout:331 write 0xd28fa68 thru 0xd29251e (0x2ab7 bytes) 2024-09-12T09:29:40.227 INFO:teuthology.orchestra.run.smithi007.stdout:333 read 0x5a7e0c7 thru 0x5a87ad4 (0x9a0e bytes) 2024-09-12T09:29:40.227 INFO:teuthology.orchestra.run.smithi007.stdout:335 punch from 0xb87a6f4 to 0xb87fee3, (0x57ef bytes) 2024-09-12T09:29:40.229 INFO:teuthology.orchestra.run.smithi007.stdout:336 trunc from 0xd29251f to 0xb0e0135 2024-09-12T09:29:40.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:40.248 INFO:teuthology.orchestra.run.smithi007.stdout:337 write 0xcea8a68 thru 0xcea9493 (0xa2c bytes) 2024-09-12T09:29:40.252 INFO:teuthology.orchestra.run.smithi007.stdout:339 punch from 0x18e5bdf to 0x18efe3a, (0xa25b bytes) 2024-09-12T09:29:40.252 INFO:teuthology.orchestra.run.smithi007.stdout:340 read 0x32ba2f thru 0x32bf80 (0x552 bytes) 2024-09-12T09:29:40.265 INFO:teuthology.orchestra.run.smithi007.stdout:341 trunc from 0xcea9494 to 0x21ec8cc 2024-09-12T09:29:40.296 INFO:teuthology.orchestra.run.smithi007.stdout:342 write 0xac6891a thru 0xac75b3e (0xd225 bytes) 2024-09-12T09:29:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:343 read 0x55f5367 thru 0x55fd193 (0x7e2d bytes) 2024-09-12T09:29:40.301 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xe6ac5be 2024-09-12T09:29:40.306 INFO:teuthology.orchestra.run.smithi007.stdout:344 trunc from 0xac75b3f to 0xe6ac5be 2024-09-12T09:29:40.310 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:40.306+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:40.327 INFO:teuthology.orchestra.run.smithi007.stdout:346 trunc from 0xe6ac5be to 0x62d4a81 2024-09-12T09:29:40.349 INFO:teuthology.orchestra.run.smithi007.stdout:347 punch from 0x47ef29d to 0x47f50af, (0x5e12 bytes) 2024-09-12T09:29:40.349 INFO:teuthology.orchestra.run.smithi007.stdout:348 write 0x541a34 thru 0x548f22 (0x74ef bytes) 2024-09-12T09:29:40.365 INFO:teuthology.orchestra.run.smithi007.stdout:349 trunc from 0x62d4a81 to 0xcb903a5 2024-09-12T09:29:40.376 INFO:teuthology.orchestra.run.smithi007.stdout:350 write 0x5e94ee3 thru 0x5e94fb8 (0xd6 bytes) 2024-09-12T09:29:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:352 punch from 0x18207f8 to 0x18293ce, (0x8bd6 bytes) 2024-09-12T09:29:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:353 read 0xb55cc57 thru 0xb55f84e (0x2bf8 bytes) 2024-09-12T09:29:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:355 punch from 0x9fa1cd2 to 0x9fab751, (0x9a7f bytes) 2024-09-12T09:29:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.381 INFO:teuthology.orchestra.run.smithi007.stdout:357 read 0x74def1 thru 0x75db88 (0xfc98 bytes) 2024-09-12T09:29:40.390 INFO:teuthology.orchestra.run.smithi007.stdout:358 trunc from 0xcb903a5 to 0x7f07bd8 2024-09-12T09:29:40.401 INFO:teuthology.orchestra.run.smithi007.stdout:360 write 0x70823c0 thru 0x708f818 (0xd459 bytes) 2024-09-12T09:29:40.405 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:40.405 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:40.405 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:40.405 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:40.405 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:40.408 INFO:teuthology.orchestra.run.smithi007.stdout:363 punch from 0x4747adf to 0x47570f7, (0xf618 bytes) 2024-09-12T09:29:40.410 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:40.410+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:40.415 INFO:teuthology.orchestra.run.smithi007.stdout:364 trunc from 0x7f07bd8 to 0x354fc21 2024-09-12T09:29:40.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:40.460 INFO:teuthology.orchestra.run.smithi007.stdout:365 write 0x8a0cc8c thru 0x8a114ee (0x4863 bytes) 2024-09-12T09:29:40.480 INFO:teuthology.orchestra.run.smithi007.stdout:366 trunc from 0x8a114ef to 0x2405a1b 2024-09-12T09:29:40.502 INFO:teuthology.orchestra.run.smithi007.stdout:368 read 0xd2f4f1 thru 0xd3b2da (0xbdea bytes) 2024-09-12T09:29:40.511 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:40.510+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:40.525 INFO:teuthology.orchestra.run.smithi007.stdout:369 write 0x5fe2f32 thru 0x5fecd26 (0x9df5 bytes) 2024-09-12T09:29:40.531 INFO:teuthology.orchestra.run.smithi007.stdout:370 read 0x3df73 thru 0x42b77 (0x4c05 bytes) 2024-09-12T09:29:40.532 INFO:teuthology.orchestra.run.smithi007.stdout:373 write 0x5e28344 thru 0x5e31d12 (0x99cf bytes) 2024-09-12T09:29:40.536 INFO:teuthology.orchestra.run.smithi007.stdout:374 read 0x501b61d thru 0x5022cc4 (0x76a8 bytes) 2024-09-12T09:29:40.552 INFO:teuthology.orchestra.run.smithi007.stdout:376 trunc from 0x5fecd27 to 0xc8f1f8c 2024-09-12T09:29:40.566 INFO:teuthology.orchestra.run.smithi007.stdout:377 write 0x24c941d thru 0x24d7ee2 (0xeac6 bytes) 2024-09-12T09:29:40.574 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.592 INFO:teuthology.orchestra.run.smithi007.stdout:379 writesame 0xdb7dd8d thru 0xdb8da0f data_size 0x1(0xfc83 bytes) 2024-09-12T09:29:40.592 INFO:teuthology.orchestra.run.smithi007.stdout:381 write 0x9f7de9e thru 0x9f83a31 (0x5b94 bytes) 2024-09-12T09:29:40.598 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:40.598 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.453892766999999, 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090235599999999999, 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142640+0000", 2024-09-12T09:29:40.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142641+0000", 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7600000000000006e-07 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142791+0000", 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014992299999999999 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142808+0000", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6946e-05 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.142818+0000", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6870000000000006e-06 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146487+0000", 2024-09-12T09:29:40.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036697040000000002 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146550+0000", 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3001000000000002e-05 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.146656+0000", 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010525 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.148650+0000", 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019946899999999999 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151520+0000", 2024-09-12T09:29:40.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028695980000000001 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151608+0000", 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8269000000000005e-05 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.151664+0000", 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5516000000000002e-05 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.353516565, 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097954749999999997, 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.606 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 43 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243016+0000", 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243019+0000", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5830000000000001e-06 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243032+0000", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3273e-05 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243059+0000", 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6466999999999999e-05 2024-09-12T09:29:40.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243074+0000", 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5780000000000001e-05 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248093+0000", 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050182890000000004 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248239+0000", 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014628199999999999 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.248459+0000", 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220125 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252338+0000", 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038786530000000001 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252513+0000", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017509900000000001 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252680+0000", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167135 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252812+0000", 2024-09-12T09:29:40.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013178899999999999 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.35342064, 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013226388, 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:40.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243113+0000", 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243190+0000", 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7043000000000001e-05 2024-09-12T09:29:40.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243204+0000", 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3587000000000001e-05 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243219+0000", 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4943000000000001e-05 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249826+0000", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006606776 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249962+0000", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013611499999999999 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.250284+0000", 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032211799999999998 2024-09-12T09:29:40.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253909+0000", 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036251310000000002 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256018+0000", 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021093829999999998 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256216+0000", 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019755900000000001 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256339+0000", 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012262899999999999 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.353261477, 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013432342, 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243272+0000", 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6599999999999994e-07 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243281+0000", 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8240000000000006e-06 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243295+0000", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.377e-05 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243303+0000", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4360000000000002e-06 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251352+0000", 2024-09-12T09:29:40.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080488069999999998 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251497+0000", 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014492199999999999 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251707+0000", 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000209946 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256473+0000", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047662 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256494+0000", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256593+0000", 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8301999999999995e-05 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256704+0000", 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111079 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:40.625 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.353212974000002, 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014730705, 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 46 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243321+0000", 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.47e-07 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243329+0000", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8059999999999995e-06 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243338+0000", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8400000000000007e-06 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243346+0000", 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5440000000000001e-06 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252951+0000", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096052099999999994 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253091+0000", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013976900000000001 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253286+0000", 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019547499999999999 2024-09-12T09:29:40.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256852+0000", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035660039999999998 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257859+0000", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010067539999999999 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257936+0000", 2024-09-12T09:29:40.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7156999999999993e-05 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258051+0000", 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114299 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.353157884000002, 2024-09-12T09:29:40.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015106299, 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 47 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:40.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243376+0000", 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3300000000000001e-07 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243454+0000", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8106000000000001e-05 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243468+0000", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.375e-05 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243476+0000", 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6230000000000003e-06 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253981+0000", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010505067999999999 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254222+0000", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000240122 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254684+0000", 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046238699999999998 2024-09-12T09:29:40.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258069+0000", 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033854919999999999 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258205+0000", 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013541900000000001 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258377+0000", 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017185900000000001 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258481+0000", 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010464 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:40.637 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.353039225, 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018308963000000001, 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 48 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243495+0000", 2024-09-12T09:29:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9999999999999996e-07 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243501+0000", 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-06 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243518+0000", 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7073000000000001e-05 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243526+0000", 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7770000000000002e-06 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256954+0000", 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013428383 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257083+0000", 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892900000000001 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257284+0000", 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020067500000000001 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261519+0000", 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042348539999999997 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261670+0000", 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015173899999999999 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261742+0000", 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1476000000000005e-05 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261803+0000", 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0956999999999999e-05 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.352917723000001, 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018360113000000001, 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.645 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243617+0000", 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.657e-06 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243629+0000", 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.242e-05 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243650+0000", 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243665+0000", 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4163e-05 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258596+0000", 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014931444 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258819+0000", 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022252799999999999 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.259193+0000", 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037449800000000002 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261650+0000", 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024574620000000001 2024-09-12T09:29:40.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261818+0000", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016791500000000001 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261924+0000", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105213 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261975+0000", 2024-09-12T09:29:40.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1722999999999999e-05 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.352785040000001, 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020862525999999999, 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243750+0000", 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.626e-06 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243917+0000", 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167193 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243947+0000", 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0145999999999999e-05 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243969+0000", 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1654e-05 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260058+0000", 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016089882999999999 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260195+0000", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013658600000000001 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260404+0000", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020855900000000001 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264456+0000", 2024-09-12T09:29:40.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040527699999999998 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264476+0000", 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9823999999999999e-05 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264547+0000", 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1116000000000001e-05 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264610+0000", 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3169000000000005e-05 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.352528308, 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020895776000000001, 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244006+0000", 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5770000000000001e-06 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244018+0000", 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1636e-05 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244037+0000", 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9360000000000001e-05 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244074+0000", 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6303000000000001e-05 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262033+0000", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017959040999999999 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262203+0000", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017056300000000001 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262414+0000", 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021082800000000001 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264629+0000", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002215416 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264706+0000", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6667000000000003e-05 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264843+0000", 2024-09-12T09:29:40.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136852 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264900+0000", 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7533e-05 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.664 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.352385409, 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021340904000000001, 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52 2024-09-12T09:29:40.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244149+0000", 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5629999999999999e-06 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244161+0000", 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2106999999999999e-05 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244191+0000", 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0162999999999999e-05 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244211+0000", 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.995e-05 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263126+0000", 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018914665000000001 2024-09-12T09:29:40.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263373+0000", 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024669200000000002 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263771+0000", 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039814700000000001 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265200+0000", 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001428728 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265241+0000", 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1350000000000002e-05 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265382+0000", 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141209 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265488+0000", 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010632999999999999 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.329951575999999, 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010003580999999999, 2024-09-12T09:29:40.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 56 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:40.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266581+0000", 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266583+0000", 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5239999999999999e-06 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266708+0000", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012525899999999999 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266730+0000", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2206000000000001e-05 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.266747+0000", 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509999999999999e-05 2024-09-12T09:29:40.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.269973+0000", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032259889999999999 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.270123+0000", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014996000000000001 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.270303+0000", 2024-09-12T09:29:40.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000180318 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.272397+0000", 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020935709999999998 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276353+0000", 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039558589999999999 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276482+0000", 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012954199999999999 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.276585+0000", 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102843 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.092113036000001, 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0095335880000000008, 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 74 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504420+0000", 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504421+0000", 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3e-07 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504444+0000", 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2979000000000001e-05 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504461+0000", 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7520000000000002e-05 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504469+0000", 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.588e-06 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.511828+0000", 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073590529999999999 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.511902+0000", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4140999999999994e-05 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512004+0000", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010264600000000001 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513829+0000", 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018242200000000001 2024-09-12T09:29:40.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513879+0000", 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9863e-05 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513919+0000", 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0382000000000001e-05 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513953+0000", 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4566000000000001e-05 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.092047676, 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099651919999999995, 2024-09-12T09:29:40.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 75 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504485+0000", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504486+0000", 2024-09-12T09:29:40.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504605+0000", 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118882 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504634+0000", 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9141000000000001e-05 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504647+0000", 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2661e-05 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512339+0000", 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076925769999999999 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512404+0000", 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4318000000000003e-05 2024-09-12T09:29:40.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.512503+0000", 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9037999999999996e-05 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513842+0000", 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001339886 2024-09-12T09:29:40.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514385+0000", 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054201900000000003 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514423+0000", 2024-09-12T09:29:40.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8306000000000002e-05 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514450+0000", 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.764e-05 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:40.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.091869083999999, 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010114253, 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 76 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504664+0000", 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504665+0000", 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504675+0000", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0155e-05 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504686+0000", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1034e-05 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.504695+0000", 2024-09-12T09:29:40.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.772e-06 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513322+0000", 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086273440000000003 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513382+0000", 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0507000000000002e-05 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.513483+0000", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010040299999999999 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514232+0000", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074900599999999998 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514714+0000", 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00048209099999999998 2024-09-12T09:29:40.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514753+0000", 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9110999999999999e-05 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.514778+0000", 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5068000000000001e-05 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:90 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733529794999999, 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010637117, 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 90 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863003+0000", 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863005+0000", 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7129999999999999e-06 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863170+0000", 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016529899999999999 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863194+0000", 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4307e-05 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863212+0000", 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7163000000000002e-05 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.865786+0000", 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025744240000000001 2024-09-12T09:29:40.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.865911+0000", 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012526200000000001 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.866106+0000", 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019473900000000001 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869618+0000", 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035115490000000001 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873348+0000", 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037307030000000001 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873522+0000", 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017342900000000001 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873640+0000", 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118529 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733282796000001, 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010820224999999999, 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 91 2024-09-12T09:29:40.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863250+0000", 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863252+0000", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5659999999999999e-06 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863262+0000", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-06 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863285+0000", 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3156999999999999e-05 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863305+0000", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0656e-05 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868464+0000", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051589649999999997 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868586+0000", 2024-09-12T09:29:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121766 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.868800+0000", 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021344499999999999 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873803+0000", 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050035089999999997 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873936+0000", 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013331199999999999 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874014+0000", 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7906999999999998e-05 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874070+0000", 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6072000000000001e-05 2024-09-12T09:29:40.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733170270999999, 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010847973, 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 92 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863364+0000", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863375+0000", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1274000000000001e-05 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863399+0000", 2024-09-12T09:29:40.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3243000000000001e-05 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863414+0000", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5653000000000001e-05 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869692+0000", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006277721 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869816+0000", 2024-09-12T09:29:40.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012430300000000001 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870084+0000", 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026783200000000002 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873917+0000", 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038328450000000001 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874082+0000", 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557200000000001 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874142+0000", 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.931e-05 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874211+0000", 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8869999999999996e-05 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733087261000001, 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012610355, 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 93 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:40.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863447+0000", 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4270000000000001e-06 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863601+0000", 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015352900000000001 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863623+0000", 2024-09-12T09:29:40.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2506000000000001e-05 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863639+0000", 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6347000000000001e-05 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870855+0000", 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072158680000000003 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870984+0000", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892599999999999 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.871189+0000", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020435200000000001 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874344+0000", 2024-09-12T09:29:40.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031557970000000001 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875801+0000", 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001456684 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875949+0000", 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014783600000000001 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876056+0000", 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107083 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.732858876000002, 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014766520999999999, 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:40.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 94 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863676+0000", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.64e-06 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863686+0000", 2024-09-12T09:29:40.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0757000000000001e-05 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863706+0000", 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9343000000000001e-05 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863721+0000", 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4946999999999999e-05 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874405+0000", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010684129000000001 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874528+0000", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012332599999999999 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874725+0000", 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019663600000000001 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876165+0000", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001440621 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878296+0000", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021301430000000001 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878380+0000", 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4462999999999994e-05 2024-09-12T09:29:40.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878441+0000", 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0516000000000001e-05 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:40.730 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:40.733 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:40.730+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:40.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:383 write 0xe7ab6b8 thru 0xe7b9d03 (0xe64c bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:384 punch from 0xe44bef4 to 0xe45b1f9, (0xf305 bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:386 read 0x3979b36 thru 0x39845f0 (0xaabb bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:389 read 0x47b5b71 thru 0x47b7ba6 (0x2036 bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:390 punch from 0x3696de3 to 0x36a5af0, (0xed0d bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:391 writesame 0x2845557 thru 0x284a4fa data_size 0x6(0x4fa4 bytes) 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.735 INFO:teuthology.orchestra.run.smithi007.stdout:393 read 0xceec59e thru 0xcef3da4 (0x7807 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:394 writesame 0xeb7e1d4 thru 0xeb85a4b data_size 0x3c(0x7878 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:395 punch from 0xeac96c9 to 0xead1449, (0x7d80 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:396 read 0x582e1ef thru 0x5835269 (0x707b bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:398 punch from 0xa2d2357 to 0xa2dd1b9, (0xae62 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:399 read 0x2140838 thru 0x2140b16 (0x2df bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:400 trunc from 0xeb85a4c to 0xaf8ecab 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:401 write 0xd613a47 thru 0xd6140cf (0x689 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:402 punch from 0xc6a2c62 to 0xc6aed52, (0xc0f0 bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:405 read 0x1d3e200 thru 0x1d42b7e (0x497f bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:406 write 0xa49f9f6 thru 0xa4a4cc4 (0x52cf bytes) 2024-09-12T09:29:40.736 INFO:teuthology.orchestra.run.smithi007.stdout:408 read 0x1e13b1a thru 0x1e18166 (0x464d bytes) 2024-09-12T09:29:40.737 INFO:teuthology.orchestra.run.smithi007.stdout:409 trunc from 0xd6140d0 to 0x8456707 2024-09-12T09:29:40.753 INFO:teuthology.orchestra.run.smithi007.stdout:410 read 0x7731f4b thru 0x7737bed (0x5ca3 bytes) 2024-09-12T09:29:40.753 INFO:teuthology.orchestra.run.smithi007.stdout:411 write 0x47871c4 thru 0x478d4e4 (0x6321 bytes) 2024-09-12T09:29:40.759 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xed57061 2024-09-12T09:29:40.773 INFO:teuthology.orchestra.run.smithi007.stdout:414 trunc from 0x8456707 to 0xed57061 2024-09-12T09:29:40.783 INFO:teuthology.orchestra.run.smithi007.stdout:415 clone 1 order 25 su 2097152 sc 14 2024-09-12T09:29:40.832 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:40.834+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:40.872 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:40.872 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:40.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:40.933 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:40.930+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T09:29:41.034 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:41.030+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:41.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T09:29:41.060 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:41.061 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:41.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:41.171 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:41.170+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:41.236 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:41.234+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5749993089999998, 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064694160000000004, 2024-09-12T09:29:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688548+0000", 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688551+0000", 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.959e-06 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688564+0000", 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2822e-05 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688589+0000", 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4876999999999999e-05 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688601+0000", 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.181e-05 2024-09-12T09:29:41.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688688+0000", 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7119000000000004e-05 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.688696+0000", 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3930000000000008e-06 2024-09-12T09:29:41.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.691882+0000", 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031855379999999999 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.692106+0000", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022441 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.692449+0000", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034254799999999997 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.693624+0000", 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011752449999999999 2024-09-12T09:29:41.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.694902+0000", 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012784179999999999 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.694951+0000", 2024-09-12T09:29:41.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8869000000000003e-05 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.695017+0000", 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6408000000000005e-05 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:825 3.1b 3:dc31104e:::rbd_data.1191192d73a0.0000000000000013:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.074692+0000", 2024-09-12T09:29:41.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.188855631, 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075961869999999999, 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 825 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074692+0000", 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074692+0000", 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074693+0000", 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6299999999999993e-07 2024-09-12T09:29:41.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074696+0000", 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.681e-06 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074706+0000", 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2520000000000005e-06 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074710+0000", 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074759+0000", 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8748000000000002e-05 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.074859+0000", 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9862e-05 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.076075+0000", 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001215846 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.082213+0000", 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061383469999999997 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.082225+0000", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1525000000000001e-05 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.082255+0000", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0614999999999997e-05 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.082288+0000", 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2589000000000003e-05 2024-09-12T09:29:41.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:851 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.123774+0000", 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.1397737379999999, 2024-09-12T09:29:41.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0041379010000000003, 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 851 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.281 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123774+0000", 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123774+0000", 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123775+0000", 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.457e-06 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123782+0000", 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1130000000000003e-06 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123796+0000", 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3968000000000001e-05 2024-09-12T09:29:41.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123803+0000", 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123840+0000", 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6550000000000001e-05 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.123886+0000", 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6121000000000001e-05 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.124078+0000", 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019254600000000001 2024-09-12T09:29:41.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.124661+0000", 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000582809 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.127852+0000", 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031913250000000001 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.127885+0000", 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2641000000000001e-05 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.127912+0000", 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6528000000000001e-05 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:860 3.17 3:eee2b7bf:::rbd_data.1191192d73a0.0000000000000027:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.146738+0000", 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.11680929, 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044716390000000003, 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 860 2024-09-12T09:29:41.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146738+0000", 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146738+0000", 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146739+0000", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146742+0000", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5439999999999999e-06 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146752+0000", 2024-09-12T09:29:41.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9240000000000002e-06 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146759+0000", 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9489999999999999e-06 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146822+0000", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2998999999999998e-05 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.146918+0000", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6069999999999993e-05 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.147042+0000", 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012409399999999999 2024-09-12T09:29:41.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.147685+0000", 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00064261400000000001 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.151139+0000", 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034538870000000001 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.151170+0000", 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0871000000000003e-05 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.151210+0000", 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9937000000000001e-05 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:977 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.328817+0000", 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.93473041499999998, 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003778834, 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 977 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328817+0000", 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328817+0000", 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328820+0000", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.196e-06 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328833+0000", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2384999999999999e-05 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328865+0000", 2024-09-12T09:29:41.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2079000000000002e-05 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.328879+0000", 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4079e-05 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.329044+0000", 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016506800000000001 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.329329+0000", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028552800000000001 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.330458+0000", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001129101 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.332368+0000", 2024-09-12T09:29:41.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001909984 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.332400+0000", 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1732e-05 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.332493+0000", 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2676000000000005e-05 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.332596+0000", 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103006 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:979 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.335173+0000", 2024-09-12T09:29:41.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.92837399200000004, 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004885983, 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 979 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335173+0000", 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335173+0000", 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335175+0000", 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.159e-06 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335180+0000", 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335194+0000", 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3713000000000001e-05 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335202+0000", 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0399999999999993e-06 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335285+0000", 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3197999999999999e-05 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.335433+0000", 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014805600000000001 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.336057+0000", 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062436499999999999 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.337492+0000", 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014351360000000001 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.339958+0000", 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024656600000000002 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.340017+0000", 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8501000000000001e-05 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.340059+0000", 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2848e-05 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1014 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.417369+0000", 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84617802799999997, 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040080430000000002, 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1014 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417369+0000", 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417369+0000", 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417373+0000", 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7069999999999998e-06 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417418+0000", 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5442999999999998e-05 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417457+0000", 2024-09-12T09:29:41.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8016000000000003e-05 2024-09-12T09:29:41.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417490+0000", 2024-09-12T09:29:41.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3526999999999999e-05 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417660+0000", 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017007900000000001 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.417946+0000", 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028631099999999998 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.419098+0000", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011512580000000001 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.420968+0000", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018705760000000001 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.421313+0000", 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034484899999999999 2024-09-12T09:29:41.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.421347+0000", 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3521e-05 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.421377+0000", 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0756000000000002e-05 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1063 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.554093+0000", 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.709454535, 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033388089999999999, 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1063 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554093+0000", 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554093+0000", 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554097+0000", 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3370000000000001e-06 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554115+0000", 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7363e-05 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554153+0000", 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8053999999999999e-05 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554172+0000", 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9746e-05 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554345+0000", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017310900000000001 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.554634+0000", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028894100000000002 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.555415+0000", 2024-09-12T09:29:41.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078027000000000001 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.557221+0000", 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018060400000000001 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.557269+0000", 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7873000000000001e-05 2024-09-12T09:29:41.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.557358+0000", 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9103000000000004e-05 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.557432+0000", 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3973000000000005e-05 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1072 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.567828+0000", 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.69571959800000005, 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00396891, 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1072 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567828+0000", 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567828+0000", 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567830+0000", 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.317e-06 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567842+0000", 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1567e-05 2024-09-12T09:29:41.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567872+0000", 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0502999999999999e-05 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.567887+0000", 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4552999999999999e-05 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.568037+0000", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015039200000000001 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.568311+0000", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027386799999999997 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.569447+0000", 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001135622 2024-09-12T09:29:41.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.571410+0000", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019630709999999998 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.571638+0000", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022842199999999999 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.571729+0000", 2024-09-12T09:29:41.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1396000000000004e-05 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.571797+0000", 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7199000000000005e-05 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1074 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.579448+0000", 2024-09-12T09:29:41.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.68409897500000005, 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036591290000000001, 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1074 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579448+0000", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579448+0000", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579452+0000", 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8940000000000003e-06 2024-09-12T09:29:41.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579471+0000", 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899999999999999e-05 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579514+0000", 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2661999999999999e-05 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579533+0000", 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8753000000000002e-05 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.579727+0000", 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194506 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.580025+0000", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029798800000000001 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.580832+0000", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080699999999999999 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.582680+0000", 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018474100000000001 2024-09-12T09:29:41.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.582813+0000", 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000133332 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.582980+0000", 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016747499999999999 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.583108+0000", 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012720900000000001 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1083 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.594039+0000", 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.66950791099999996, 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039016739999999999, 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1083 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594039+0000", 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594039+0000", 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594043+0000", 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.45e-06 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594054+0000", 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1533e-05 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594086+0000", 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1420000000000001e-05 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594100+0000", 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4569000000000001e-05 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594250+0000", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014990899999999999 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.594521+0000", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000270485 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.595613+0000", 2024-09-12T09:29:41.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001091678 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.597717+0000", 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021040519999999999 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.597765+0000", 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8856000000000001e-05 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.597868+0000", 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102792 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.597941+0000", 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929999999999995e-05 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1085 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.603478+0000", 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.66006899200000002, 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035236569999999999, 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1085 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603478+0000", 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603478+0000", 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603483+0000", 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.16e-06 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603499+0000", 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6256000000000002e-05 2024-09-12T09:29:41.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603538+0000", 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9223e-05 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603557+0000", 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8723e-05 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.603719+0000", 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162246 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.604003+0000", 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028353800000000001 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.604821+0000", 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000818307 2024-09-12T09:29:41.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.606760+0000", 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019388389999999999 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.606831+0000", 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1045999999999997e-05 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.606926+0000", 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5272999999999994e-05 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.607002+0000", 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6045999999999997e-05 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1094 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.616468+0000", 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.64707952000000002, 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037289649999999999, 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1094 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616468+0000", 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616468+0000", 2024-09-12T09:29:41.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616470+0000", 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6129999999999998e-06 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616483+0000", 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2387e-05 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616533+0000", 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0065999999999998e-05 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616550+0000", 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7363999999999998e-05 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616666+0000", 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116035 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.616827+0000", 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000160929 2024-09-12T09:29:41.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.617416+0000", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058873900000000002 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.619088+0000", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001671748 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.620038+0000", 2024-09-12T09:29:41.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095009900000000004 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.620128+0000", 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.620197+0000", 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9295999999999995e-05 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1096 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.624042+0000", 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.63950501000000004, 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035246570000000001, 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1096 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624042+0000", 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624042+0000", 2024-09-12T09:29:41.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624046+0000", 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3699999999999999e-06 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624057+0000", 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1063999999999999e-05 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624085+0000", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8646e-05 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624102+0000", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6722999999999999e-05 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624237+0000", 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013467199999999999 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.624386+0000", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014895199999999999 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.625012+0000", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062648900000000004 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.626676+0000", 2024-09-12T09:29:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001663798 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.627407+0000", 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073048799999999995 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.627499+0000", 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2619000000000002e-05 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.627567+0000", 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7836000000000006e-05 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1111 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.669390+0000", 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.59415738399999996, 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034736739999999999, 2024-09-12T09:29:41.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1111 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669390+0000", 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669390+0000", 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669392+0000", 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669403+0000", 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0933e-05 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669433+0000", 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9824000000000001e-05 2024-09-12T09:29:41.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669448+0000", 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4783000000000001e-05 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669560+0000", 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112392 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.669713+0000", 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152719 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.670115+0000", 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000402137 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.671527+0000", 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001411806 2024-09-12T09:29:41.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.672693+0000", 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001166708 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.672783+0000", 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9242999999999999e-05 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.672864+0000", 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079000000000001e-05 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1119 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.686941+0000", 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.576606226, 2024-09-12T09:29:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034110450000000001, 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1119 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.686941+0000", 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.686941+0000", 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.686946+0000", 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4800000000000003e-06 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.686965+0000", 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9147000000000002e-05 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.687011+0000", 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5825999999999998e-05 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.687031+0000", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0363000000000001e-05 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.687183+0000", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152025 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.687359+0000", 2024-09-12T09:29:41.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017616600000000001 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.687798+0000", 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000438577 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.689520+0000", 2024-09-12T09:29:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001722337 2024-09-12T09:29:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.690180+0000", 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000659608 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.690277+0000", 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7306000000000007e-05 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.690352+0000", 2024-09-12T09:29:41.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5210000000000007e-05 2024-09-12T09:29:41.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1128 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.700118+0000", 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56342979199999998, 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003608034, 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1128 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700118+0000", 2024-09-12T09:29:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700118+0000", 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700120+0000", 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-06 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700134+0000", 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4387e-05 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700164+0000", 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-05 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700179+0000", 2024-09-12T09:29:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.434e-05 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700315+0000", 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013623199999999999 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.700468+0000", 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015316200000000001 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.701066+0000", 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00059776600000000001 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.702683+0000", 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016176210000000001 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.703554+0000", 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087112099999999998 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.703644+0000", 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9189000000000003e-05 2024-09-12T09:29:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.703726+0000", 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2006000000000001e-05 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1130 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.707492+0000", 2024-09-12T09:29:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.55605518300000001, 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038084780000000001, 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1130 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707492+0000", 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707492+0000", 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707495+0000", 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3800000000000001e-06 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707506+0000", 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1749999999999999e-05 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707537+0000", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0505999999999999e-05 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707552+0000", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4927e-05 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707673+0000", 2024-09-12T09:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121166 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.707828+0000", 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015489900000000001 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.708441+0000", 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061360099999999997 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.710112+0000", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001670831 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711139+0000", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001026323 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711232+0000", 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3329e-05 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711301+0000", 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8765999999999999e-05 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1132 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.730420+0000", 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53312691199999995, 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038097180000000001, 2024-09-12T09:29:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1132 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730420+0000", 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730420+0000", 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730425+0000", 2024-09-12T09:29:41.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2230000000000001e-06 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730444+0000", 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9556999999999999e-05 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730487+0000", 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2518999999999997e-05 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730508+0000", 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0959999999999999e-05 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730648+0000", 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014063199999999999 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.730821+0000", 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000172213 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.731426+0000", 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060574499999999998 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.733242+0000", 2024-09-12T09:29:41.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001815243 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734060+0000", 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081824100000000004 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734155+0000", 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5551999999999996e-05 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734230+0000", 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4833000000000007e-05 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1161 3.1f 3:fd5a894f:::rbd_object_map.1191192d73a0:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:41.026432+0000", 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.237115511, 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00491035, 2024-09-12T09:29:41.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1161 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026432+0000", 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026432+0000", 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026434+0000", 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5270000000000001e-06 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026445+0000", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0934999999999999e-05 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026469+0000", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3907999999999999e-05 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026481+0000", 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1647999999999999e-05 2024-09-12T09:29:41.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026588+0000", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010676 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.026600+0000", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2217e-05 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.031147+0000", 2024-09-12T09:29:41.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045472029999999997 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.031232+0000", 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5267999999999997e-05 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:41.031342+0000", 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010988399999999999 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.410 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:41.411 INFO:teuthology.orchestra.run.smithi007.stdout:418 write 0x9d3906d thru 0x9d478b8 (0xe84c bytes) 2024-09-12T09:29:41.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:41.413 INFO:teuthology.orchestra.run.smithi007.stdout:419 read 0x3e8e40b thru 0x3e90387 (0x1f7d bytes) 2024-09-12T09:29:41.413 INFO:teuthology.orchestra.run.smithi007.stdout:420 read 0xb34b785 thru 0xb34e3c6 (0x2c42 bytes) 2024-09-12T09:29:41.413 INFO:teuthology.orchestra.run.smithi007.stdout:422 clone 2 order 25 su 1048576 sc 10 2024-09-12T09:29:41.414 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:41.410+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:41.513 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:41.510+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:41.585 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:41.585 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:41.585 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:41.585 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:41.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:41.644 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:41.642+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:41.714 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:41.710+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:41.769 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:41.770 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:41.770 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:41.770 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:41.770 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:41.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:41.827 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:41.826+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:41.916 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:41.914+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:41.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4217.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:24.956568+0000", 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.996871274, 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019952949999999998, 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956568+0000", 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956568+0000", 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956568+0000", 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1300000000000001e-07 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956577+0000", 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4970000000000001e-06 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956607+0000", 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0536e-05 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956626+0000", 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9130000000000001e-05 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956770+0000", 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000143089 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.956806+0000", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6906999999999997e-05 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.958456+0000", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016496830000000001 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:24.958563+0000", 2024-09-12T09:29:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010704000000000001 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4217.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:25.962986+0000", 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.990452791999999, 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002062445, 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.962986+0000", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.962986+0000", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.962987+0000", 2024-09-12T09:29:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.963005+0000", 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8130000000000001e-05 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.963041+0000", 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5689999999999999e-05 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.963069+0000", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7826000000000001e-05 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.963259+0000", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019027200000000001 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.963323+0000", 2024-09-12T09:29:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4269000000000004e-05 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.964968+0000", 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001644438 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:25.965049+0000", 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1036000000000001e-05 2024-09-12T09:29:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:7 3.1a e25/23 3:5cea7035:::rbd_info:head v 25'1, mlcod=0'0)", 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:35.667371+0000", 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2860683589999997, 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001920129, 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667371+0000", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667371+0000", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667371+0000", 2024-09-12T09:29:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667380+0000", 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4929999999999998e-06 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667410+0000", 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0352999999999999e-05 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667428+0000", 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7859999999999998e-05 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667566+0000", 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013792200000000001 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.667602+0000", 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6285999999999998e-05 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.669213+0000", 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001610855 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:35.669291+0000", 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7823000000000004e-05 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:36.676005+0000", 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.277434381, 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017458650000000001, 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676005+0000", 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676005+0000", 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676005+0000", 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.51e-07 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676012+0000", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676040+0000", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8459e-05 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676057+0000", 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6562000000000002e-05 2024-09-12T09:29:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676210+0000", 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015356300000000001 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.676277+0000", 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7138999999999995e-05 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.677611+0000", 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001333964 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:36.677751+0000", 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139252 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:675 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:39.809317+0000", 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1441221069999998, 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014745839999999999, 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 675 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809317+0000", 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809317+0000", 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809319+0000", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7880000000000001e-06 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809325+0000", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809337+0000", 2024-09-12T09:29:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2194e-05 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809343+0000", 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1530000000000002e-06 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809392+0000", 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8337000000000002e-05 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809485+0000", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3610999999999995e-05 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.809609+0000", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012322400000000001 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.810293+0000", 2024-09-12T09:29:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068441400000000005 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.810681+0000", 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038805099999999999 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.810742+0000", 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1105999999999994e-05 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.810792+0000", 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9546000000000003e-05 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:682 3.0 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:39.835448+0000", 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1179915970000001, 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016663579999999999, 2024-09-12T09:29:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 682 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835448+0000", 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835448+0000", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835450+0000", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0779999999999998e-06 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835456+0000", 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4320000000000004e-06 2024-09-12T09:29:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835473+0000", 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7223000000000001e-05 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835483+0000", 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9240000000000002e-06 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835562+0000", 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8370999999999999e-05 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835644+0000", 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2746999999999999e-05 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835754+0000", 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109739 2024-09-12T09:29:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.836731+0000", 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097693699999999999 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837030+0000", 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877199999999999 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837071+0000", 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1295999999999999e-05 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837114+0000", 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2839000000000001e-05 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:683 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:39.835508+0000", 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1179309059999998, 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001938988, 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 683 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835508+0000", 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835508+0000", 2024-09-12T09:29:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835509+0000", 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835515+0000", 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6829999999999996e-06 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835533+0000", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8388000000000001e-05 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.835541+0000", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0989999999999999e-06 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.836083+0000", 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054169099999999996 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.836163+0000", 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0193999999999998e-05 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.836274+0000", 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110643 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837322+0000", 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001048315 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837357+0000", 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4758000000000002e-05 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837406+0000", 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9719000000000003e-05 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.837447+0000", 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0938e-05 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:728 3.19 3:9af5fad5:::rbd_data.1191192d73a0.0000000000000006:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:39.883358+0000", 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0700815600000002, 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001537715, 2024-09-12T09:29:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 728 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883358+0000", 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883358+0000", 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883359+0000", 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.48e-06 2024-09-12T09:29:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883365+0000", 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.417e-06 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883376+0000", 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1926999999999999e-05 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883382+0000", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6930000000000004e-06 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883445+0000", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000005e-05 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883535+0000", 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9532999999999998e-05 2024-09-12T09:29:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.883734+0000", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000199216 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.884500+0000", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076599299999999997 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.884835+0000", 2024-09-12T09:29:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033465099999999999 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.884868+0000", 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3198000000000003e-05 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:39.884895+0000", 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7617e-05 2024-09-12T09:29:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.9035052590000001, 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:29:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:820 3.0 e28/23 ondisk, result = 0)", 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.061668+0000", 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.8917716599999999, 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005323169, 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061668+0000", 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061668+0000", 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061668+0000", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061672+0000", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.506e-06 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061683+0000", 2024-09-12T09:29:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1633e-05 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.061690+0000", 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3949999999999998e-06 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066862+0000", 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051719699999999997 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066869+0000", 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7049999999999999e-06 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066991+0000", 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012121999999999999 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:823 3.19 3:9e9871c0:::rbd_data.1191192d73a0.000000000000003a:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.070354+0000", 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.8830849409999999, 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001921507, 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 823 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070354+0000", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070354+0000", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070356+0000", 2024-09-12T09:29:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.632e-06 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070363+0000", 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9299999999999997e-06 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070378+0000", 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5449000000000001e-05 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070386+0000", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3359999999999997e-06 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070451+0000", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5343000000000001e-05 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070543+0000", 2024-09-12T09:29:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2193000000000003e-05 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.070813+0000", 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026950200000000002 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.071698+0000", 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00088504600000000001 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.072169+0000", 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047150000000000002 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.072216+0000", 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7024999999999998e-05 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.072276+0000", 2024-09-12T09:29:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9551e-05 2024-09-12T09:29:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1028 3.8 e28/23 3:10c33995:::rbd_header.1191192d73a0:head v 28'102, mlcod=28'100)", 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.452351+0000", 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.501088024, 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001591861, 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452351+0000", 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452351+0000", 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452354+0000", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8660000000000002e-06 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452374+0000", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9817e-05 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452413+0000", 2024-09-12T09:29:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9396e-05 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452436+0000", 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2730000000000001e-05 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452614+0000", 2024-09-12T09:29:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017799499999999999 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.452678+0000", 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4156e-05 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.453850+0000", 2024-09-12T09:29:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011714690000000001 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.453943+0000", 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1037 3.0 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2149516~18531] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.464740+0000", 2024-09-12T09:29:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.488699011, 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028763450000000002, 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1037 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464740+0000", 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464740+0000", 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464743+0000", 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6400000000000001e-06 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464770+0000", 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7682999999999999e-05 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464798+0000", 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7667000000000001e-05 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464813+0000", 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4899000000000001e-05 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.464948+0000", 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013504599999999999 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465085+0000", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000136849 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465300+0000", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021538500000000001 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.466982+0000", 2024-09-12T09:29:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001681931 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467459+0000", 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047654300000000001 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467549+0000", 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0629000000000003e-05 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467617+0000", 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7073e-05 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1039 3.0 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.485047+0000", 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4683917360000001, 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023942049999999999, 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1039 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485047+0000", 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485047+0000", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485050+0000", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6929999999999999e-06 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485062+0000", 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1766999999999999e-05 2024-09-12T09:29:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485095+0000", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3368999999999998e-05 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485112+0000", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6273000000000002e-05 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485228+0000", 2024-09-12T09:29:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011694 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485364+0000", 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135859 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485546+0000", 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018215199999999999 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.486584+0000", 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001037318 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487295+0000", 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071094500000000004 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487376+0000", 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0908999999999994e-05 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487442+0000", 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5980000000000002e-05 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1051 3.8 e28/23 3:10c33995:::rbd_header.1191192d73a0:head v 28'104, mlcod=28'102)", 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.516928+0000", 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.436510779, 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001642601, 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.516928+0000", 2024-09-12T09:29:42.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.516928+0000", 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.516931+0000", 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.503e-06 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.516949+0000", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8077e-05 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.516989+0000", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9860000000000001e-05 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.517008+0000", 2024-09-12T09:29:42.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9663000000000001e-05 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.517147+0000", 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013841799999999999 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.517185+0000", 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8127000000000003e-05 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.518491+0000", 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001306043 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.518571+0000", 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9909999999999999e-05 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1064 3.8 e28/23 3:10c33995:::rbd_header.1191192d73a0:head v 28'105, mlcod=28'103)", 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.558977+0000", 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.394462562, 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015449420000000001, 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.558977+0000", 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.558977+0000", 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.558979+0000", 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6900000000000001e-06 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.558994+0000", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4552999999999999e-05 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.559028+0000", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4597000000000001e-05 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.559047+0000", 2024-09-12T09:29:42.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8862999999999999e-05 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.559192+0000", 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014452599999999999 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.559231+0000", 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8983000000000003e-05 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.560417+0000", 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011857669999999999 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.560522+0000", 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104963 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1099 3.8 e28/23 3:10c33995:::rbd_header.1191192d73a0:head v 28'108, mlcod=28'106)", 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.636121+0000", 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.317318021, 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017001379999999999, 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636121+0000", 2024-09-12T09:29:42.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636121+0000", 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636123+0000", 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9400000000000001e-06 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636135+0000", 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636165+0000", 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9547e-05 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636179+0000", 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4375999999999999e-05 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636300+0000", 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012020599999999999 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.636346+0000", 2024-09-12T09:29:42.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6403000000000003e-05 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.637705+0000", 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001359166 2024-09-12T09:29:42.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.637821+0000", 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001161 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1120 3.8 e28/23 3:10c33995:::rbd_header.1191192d73a0:head v 28'110, mlcod=28'108)", 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.691520+0000", 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.261918834, 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001469773, 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691520+0000", 2024-09-12T09:29:42.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691520+0000", 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691523+0000", 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5339999999999998e-06 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691537+0000", 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3689999999999999e-05 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691570+0000", 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3352999999999997e-05 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691588+0000", 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8312999999999999e-05 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691737+0000", 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014881200000000001 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.691803+0000", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.692889+0000", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010859559999999999 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.692990+0000", 2024-09-12T09:29:42.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100729 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1131 3.0 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head [set-alloc-hint object_size 4194304 write_size 4194304,write 653814~21199] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.711528+0000", 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.2419115039999999, 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037796679999999999, 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1131 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711528+0000", 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711528+0000", 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711531+0000", 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711583+0000", 2024-09-12T09:29:42.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506000000000002e-05 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711616+0000", 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3083000000000002e-05 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711631+0000", 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5174000000000001e-05 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711781+0000", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149285 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.711946+0000", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164859 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712173+0000", 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022741499999999999 2024-09-12T09:29:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.713966+0000", 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017926859999999999 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.715142+0000", 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001176641 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.715237+0000", 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4517000000000003e-05 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.715307+0000", 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0481999999999994e-05 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:1134 3.0 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.734550+0000", 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.2188895399999999, 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003043394, 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1134 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734550+0000", 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:42.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734550+0000", 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734553+0000", 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6629999999999999e-06 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734569+0000", 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5634e-05 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734613+0000", 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4202999999999997e-05 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734638+0000", 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5006000000000001e-05 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734775+0000", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137269 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.734937+0000", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016185900000000001 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735099+0000", 2024-09-12T09:29:42.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161989 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736643+0000", 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001543918 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737372+0000", 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072838099999999995 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:42.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737452+0000", 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0865999999999995e-05 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737593+0000", 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014060599999999999 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:42.076 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:42.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:29:42.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:42.079 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T09:29:42.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T09:29:42.080 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:42.080 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:42.074+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:42.091 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:42.091 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:42.115 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:42.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:42.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:42.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:42.162 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone1 from 0xed57061 (overlap 0xed57061) to 0x2f77ce0 2024-09-12T09:29:42.179 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:42.174+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:42.187 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:42.187 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:42.215 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:42.215 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:42.226 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:42.226 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:42.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:42.280 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:42.278+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:42.313 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-12T09:29:42.380 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:42.378+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:42.437 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:42.437 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:42.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:42.499 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:42.498+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:42.582 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:42.578+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:42.617 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:42.617 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:42.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:29:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T09:29:42.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T09:29:42.715 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:42.714+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:42.725 INFO:teuthology.orchestra.run.smithi007.stdout:424 write 0x51d7ad thru 0x52952e (0xbd82 bytes) 2024-09-12T09:29:42.760 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:42.760 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:42.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:42.783 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:42.782+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:42.786 INFO:teuthology.orchestra.run.smithi007.stdout:426 read 0xc863f0a thru 0xc86cfd3 (0x90ca bytes) 2024-09-12T09:29:42.786 INFO:teuthology.orchestra.run.smithi007.stdout:428 read 0x2339f90 thru 0x2342d67 (0x8dd8 bytes) 2024-09-12T09:29:42.787 INFO:teuthology.orchestra.run.smithi007.stdout:430 read 0x518f0f6 thru 0x519d66c (0xe577 bytes) 2024-09-12T09:29:42.787 INFO:teuthology.orchestra.run.smithi007.stdout:432 read 0xa8fa109 thru 0xa8fe303 (0x41fb bytes) 2024-09-12T09:29:42.787 INFO:teuthology.orchestra.run.smithi007.stdout:433 read 0x9ff5f2e thru 0x9ff6c4f (0xd22 bytes) 2024-09-12T09:29:42.788 INFO:teuthology.orchestra.run.smithi007.stdout:434 punch from 0x7a7db29 to 0x7a874bd, (0x9994 bytes) 2024-09-12T09:29:42.788 INFO:teuthology.orchestra.run.smithi007.stdout:436 write 0xe78cc16 thru 0xe79ca20 (0xfe0b bytes) 2024-09-12T09:29:42.831 INFO:teuthology.orchestra.run.smithi007.stdout:439 punch from 0xecd98d1 to 0xecded78, (0x54a7 bytes) 2024-09-12T09:29:42.831 INFO:teuthology.orchestra.run.smithi007.stdout:440 read 0xe692209 thru 0xe696492 (0x428a bytes) 2024-09-12T09:29:42.831 INFO:teuthology.orchestra.run.smithi007.stdout:441 write 0x606ce66 thru 0x60765c0 (0x975b bytes) 2024-09-12T09:29:42.882 INFO:teuthology.orchestra.run.smithi007.stdout:442 read 0xd2040d7 thru 0xd210053 (0xbf7d bytes) 2024-09-12T09:29:42.883 INFO:teuthology.orchestra.run.smithi007.stdout:444 write 0x844a169 thru 0x845537f (0xb217 bytes) 2024-09-12T09:29:42.883 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:42.882+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:42.906 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:42.906 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:42.921 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:42.921 INFO:teuthology.orchestra.run.smithi007.stdout:447 punch from 0xe7a405e to 0xe7ac96f, (0x8911 bytes) 2024-09-12T09:29:42.921 INFO:teuthology.orchestra.run.smithi007.stdout:448 read 0x47f456f thru 0x47fda15 (0x94a7 bytes) 2024-09-12T09:29:42.921 INFO:teuthology.orchestra.run.smithi007.stdout:449 read 0x5ceeb29 thru 0x5cefdb7 (0x128f bytes) 2024-09-12T09:29:42.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:42.923 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:42.941 INFO:teuthology.orchestra.run.smithi007.stdout:452 trunc from 0xed57061 to 0x197e881 2024-09-12T09:29:42.984 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:42.982+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:42.985 INFO:teuthology.orchestra.run.smithi007.stdout:453 read 0x124ea96 thru 0x12590a7 (0xa612 bytes) 2024-09-12T09:29:42.998 INFO:teuthology.orchestra.run.smithi007.stdout:454 trunc from 0x197e881 to 0xd209817 2024-09-12T09:29:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:455 write 0x92d049e thru 0x92d3ba4 (0x3707 bytes) 2024-09-12T09:29:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:456 punch from 0x24d6da0 to 0x24e14d4, (0xa734 bytes) 2024-09-12T09:29:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:458 punch from 0x5fce2f5 to 0x5fd9456, (0xb161 bytes) 2024-09-12T09:29:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:459 read 0x4cad3e9 thru 0x4cb47a5 (0x73bd bytes) 2024-09-12T09:29:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:460 write 0xe2a05f6 thru 0xe2a3add (0x34e8 bytes) 2024-09-12T09:29:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:463 punch from 0x7d95728 to 0x7d98150, (0x2a28 bytes) 2024-09-12T09:29:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:465 read 0x2a6145 thru 0x2a6690 (0x54c bytes) 2024-09-12T09:29:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:467 read 0x895286c thru 0x895a9ff (0x8194 bytes) 2024-09-12T09:29:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:468 trunc from 0xe2a3ade to 0x5c3dc85 2024-09-12T09:29:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:474 writesame 0x9396e4 thru 0x942570 data_size 0x1(0x8e8d bytes) 2024-09-12T09:29:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.083 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:43.084 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:43.082+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:43.130 INFO:teuthology.orchestra.run.smithi007.stdout:475 write 0xe2ccdf2 thru 0xe2dc544 (0xf753 bytes) 2024-09-12T09:29:43.132 INFO:teuthology.orchestra.run.smithi007.stdout:477 read 0x820cba8 thru 0x821c56d (0xf9c6 bytes) 2024-09-12T09:29:43.133 INFO:teuthology.orchestra.run.smithi007.stdout:478 read 0x6850319 thru 0x685792f (0x7617 bytes) 2024-09-12T09:29:43.134 INFO:teuthology.orchestra.run.smithi007.stdout:479 punch from 0x92e3dae to 0x92ef12e, (0xb380 bytes) 2024-09-12T09:29:43.134 INFO:teuthology.orchestra.run.smithi007.stdout:480 writesame 0xddbb5a7 thru 0xddc159b data_size 0x11(0x5ff5 bytes) 2024-09-12T09:29:43.136 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.137 INFO:teuthology.orchestra.run.smithi007.stdout:482 write 0xcea1b2e thru 0xceab8ba (0x9d8d bytes) 2024-09-12T09:29:43.139 INFO:teuthology.orchestra.run.smithi007.stdout:483 read 0x3b33f03 thru 0x3b376e8 (0x37e6 bytes) 2024-09-12T09:29:43.140 INFO:teuthology.orchestra.run.smithi007.stdout:486 read 0x129854 thru 0x12eb5e (0x530b bytes) 2024-09-12T09:29:43.161 INFO:teuthology.orchestra.run.smithi007.stdout:487 trunc from 0xe2dc545 to 0x28ca754 2024-09-12T09:29:43.179 INFO:teuthology.orchestra.run.smithi007.stdout:489 read 0x697975 thru 0x6a67f5 (0xee81 bytes) 2024-09-12T09:29:43.181 INFO:teuthology.orchestra.run.smithi007.stdout:490 punch from 0x43b177 to 0x448429, (0xd2b2 bytes) 2024-09-12T09:29:43.181 INFO:teuthology.orchestra.run.smithi007.stdout:492 write 0x2857dd9 thru 0x28651d1 (0xd3f9 bytes) 2024-09-12T09:29:43.186 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:43.182+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:43.186 INFO:teuthology.orchestra.run.smithi007.stdout:493 read 0xe40550 thru 0xe42099 (0x1b4a bytes) 2024-09-12T09:29:43.186 INFO:teuthology.orchestra.run.smithi007.stdout:494 read 0x2de35c thru 0x2e1020 (0x2cc5 bytes) 2024-09-12T09:29:43.206 INFO:teuthology.orchestra.run.smithi007.stdout:496 write 0x4e96062 thru 0x4e9a3d1 (0x4370 bytes) 2024-09-12T09:29:43.220 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.220 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.232 INFO:teuthology.orchestra.run.smithi007.stdout:497 write 0x86e4f51 thru 0x86ec372 (0x7422 bytes) 2024-09-12T09:29:43.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:498 read 0x50070fb thru 0x5011a0a (0xa910 bytes) 2024-09-12T09:29:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:500 write 0x4466019 thru 0x44752b4 (0xf29c bytes) 2024-09-12T09:29:43.240 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.240 INFO:teuthology.orchestra.run.smithi007.stdout:502 read 0x651d231 thru 0x6522ff9 (0x5dc9 bytes) 2024-09-12T09:29:43.254 INFO:teuthology.orchestra.run.smithi007.stdout:503 trunc from 0x86ec373 to 0x356205e 2024-09-12T09:29:43.300 INFO:teuthology.orchestra.run.smithi007.stdout:505 write 0x85dcfdb thru 0x85e6ef8 (0x9f1e bytes) 2024-09-12T09:29:43.323 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:43.322+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:43.326 INFO:teuthology.orchestra.run.smithi007.stdout:507 writesame 0xcdb376e thru 0xcdb6845 data_size 0xc(0x30d8 bytes) 2024-09-12T09:29:43.329 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.338 INFO:teuthology.orchestra.run.smithi007.stdout:509 trunc from 0xcdb6846 to 0x95e3d8c 2024-09-12T09:29:43.358 INFO:teuthology.orchestra.run.smithi007.stdout:510 read 0x34cea2e thru 0x34dc8fb (0xdece bytes) 2024-09-12T09:29:43.385 INFO:teuthology.orchestra.run.smithi007.stdout:511 write 0xed78be2 thru 0xed83a60 (0xae7f bytes) 2024-09-12T09:29:43.387 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:43.382+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:43.389 INFO:teuthology.orchestra.run.smithi007.stdout:512 write 0x3cdd4f4 thru 0x3ceb2e0 (0xdded bytes) 2024-09-12T09:29:43.393 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.393 INFO:teuthology.orchestra.run.smithi007.stdout:515 writesame 0xde67bf thru 0xdeb97b data_size 0x9(0x51bd bytes) 2024-09-12T09:29:43.397 INFO:teuthology.orchestra.run.smithi007.stdout:517 write 0xc88a52a thru 0xc88c1a0 (0x1c77 bytes) 2024-09-12T09:29:43.399 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.400 INFO:teuthology.orchestra.run.smithi007.stdout:520 write 0x5b6e1a2 thru 0x5b6fefd (0x1d5c bytes) 2024-09-12T09:29:43.409 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.409 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.424 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:43.435 INFO:teuthology.orchestra.run.smithi007.stdout:521 trunc from 0xed83a61 to 0x3068c28 2024-09-12T09:29:43.452 INFO:teuthology.orchestra.run.smithi007.stdout:523 punch from 0x1d394ff to 0x1d3e398, (0x4e99 bytes) 2024-09-12T09:29:43.484 INFO:teuthology.orchestra.run.smithi007.stdout:524 write 0xbe34543 thru 0xbe36b28 (0x25e6 bytes) 2024-09-12T09:29:43.487 INFO:teuthology.orchestra.run.smithi007.stdout:525 read 0x6227f6 thru 0x62bfa0 (0x97ab bytes) 2024-09-12T09:29:43.488 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:43.486+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:43.489 INFO:teuthology.orchestra.run.smithi007.stdout:528 punch from 0x119869d to 0x11a2d2d, (0xa690 bytes) 2024-09-12T09:29:43.489 INFO:teuthology.orchestra.run.smithi007.stdout:529 punch from 0x11c7649 to 0x11d5c86, (0xe63d bytes) 2024-09-12T09:29:43.489 INFO:teuthology.orchestra.run.smithi007.stdout:532 writesame 0xb752682 thru 0xb7618d8 data_size 0x1(0xf257 bytes) 2024-09-12T09:29:43.536 INFO:teuthology.orchestra.run.smithi007.stdout:534 write 0x945247b thru 0x9455a14 (0x359a bytes) 2024-09-12T09:29:43.540 INFO:teuthology.orchestra.run.smithi007.stdout:535 read 0xb8e34f1 thru 0xb8e48d9 (0x13e9 bytes) 2024-09-12T09:29:43.541 INFO:teuthology.orchestra.run.smithi007.stdout:536 read 0x8e163a5 thru 0x8e20ceb (0xa947 bytes) 2024-09-12T09:29:43.562 INFO:teuthology.orchestra.run.smithi007.stdout:538 write 0xedddd20 thru 0xede7115 (0x93f6 bytes) 2024-09-12T09:29:43.566 INFO:teuthology.orchestra.run.smithi007.stdout:539 writesame 0xa4916b1 thru 0xa498fab data_size 0x1(0x78fb bytes) 2024-09-12T09:29:43.589 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:43.586+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:43.597 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.597 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.604 INFO:teuthology.orchestra.run.smithi007.stdout:540 writesame 0xe9f0381 thru 0xe9f277f data_size 0x13(0x23ff bytes) 2024-09-12T09:29:43.608 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:43.637 INFO:teuthology.orchestra.run.smithi007.stdout:542 trunc from 0xede7116 to 0x3c75359 2024-09-12T09:29:43.656 INFO:teuthology.orchestra.run.smithi007.stdout:543 read 0x35c4fe7 thru 0x35c5785 (0x79f bytes) 2024-09-12T09:29:43.677 INFO:teuthology.orchestra.run.smithi007.stdout:544 write 0x74c34e9 thru 0x74d1ec2 (0xe9da bytes) 2024-09-12T09:29:43.689 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:43.686+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:43.696 INFO:teuthology.orchestra.run.smithi007.stdout:545 write 0x9b87d07 thru 0x9b88f3a (0x1234 bytes) 2024-09-12T09:29:43.699 INFO:teuthology.orchestra.run.smithi007.stdout:546 write 0x625f747 thru 0x626b949 (0xc203 bytes) 2024-09-12T09:29:43.704 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.704 INFO:teuthology.orchestra.run.smithi007.stdout:549 read 0x452b70f thru 0x4538e98 (0xd78a bytes) 2024-09-12T09:29:43.705 INFO:teuthology.orchestra.run.smithi007.stdout:553 read 0x3ed7584 thru 0x3ee36a8 (0xc125 bytes) 2024-09-12T09:29:43.706 INFO:teuthology.orchestra.run.smithi007.stdout:554 writesame 0x20e1e5b thru 0x20eef1a data_size 0x50(0xd0c0 bytes) 2024-09-12T09:29:43.709 INFO:teuthology.orchestra.run.smithi007.stdout:555 read 0x51ff815 thru 0x520c834 (0xd020 bytes) 2024-09-12T09:29:43.710 INFO:teuthology.orchestra.run.smithi007.stdout:556 write 0x6a39e8d thru 0x6a3c9ae (0x2b22 bytes) 2024-09-12T09:29:43.735 INFO:teuthology.orchestra.run.smithi007.stdout:558 trunc from 0x9b88f3b to 0x2069799 2024-09-12T09:29:43.786 INFO:teuthology.orchestra.run.smithi007.stdout:559 trunc from 0x2069799 to 0xe89cf02 2024-09-12T09:29:43.790 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:43.786+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:43.791 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.791 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.800 INFO:teuthology.orchestra.run.smithi007.stdout:560 trunc from 0xe89cf02 to 0xe20b298 2024-09-12T09:29:43.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:43.814 INFO:teuthology.orchestra.run.smithi007.stdout:561 read 0xc8a031b thru 0xc8a125a (0xf40 bytes) 2024-09-12T09:29:43.815 INFO:teuthology.orchestra.run.smithi007.stdout:562 punch from 0x9427e75 to 0x94284b2, (0x63d bytes) 2024-09-12T09:29:43.835 INFO:teuthology.orchestra.run.smithi007.stdout:563 trunc from 0xe20b298 to 0x6d77516 2024-09-12T09:29:43.870 INFO:teuthology.orchestra.run.smithi007.stdout:564 write 0x9278565 thru 0x928360f (0xb0ab bytes) 2024-09-12T09:29:43.874 INFO:teuthology.orchestra.run.smithi007.stdout:565 punch from 0x6d3298c to 0x6d41493, (0xeb07 bytes) 2024-09-12T09:29:43.874 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.874 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:43.874 INFO:teuthology.orchestra.run.smithi007.stdout:569 read 0x6fe207c thru 0x6fe59ee (0x3973 bytes) 2024-09-12T09:29:43.875 INFO:teuthology.orchestra.run.smithi007.stdout:570 read 0x239fae9 thru 0x23a5316 (0x582e bytes) 2024-09-12T09:29:43.876 INFO:teuthology.orchestra.run.smithi007.stdout:571 punch from 0x3502a6d to 0x350582b, (0x2dbe bytes) 2024-09-12T09:29:43.876 INFO:teuthology.orchestra.run.smithi007.stdout:572 punch from 0x3ca3b53 to 0x3cb1109, (0xd5b6 bytes) 2024-09-12T09:29:43.890 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:43.886+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:43.901 INFO:teuthology.orchestra.run.smithi007.stdout:573 write 0xeb0c21a thru 0xeb0c417 (0x1fe bytes) 2024-09-12T09:29:43.903 INFO:teuthology.orchestra.run.smithi007.stdout:574 write 0x708fdb8 thru 0x7097061 (0x72aa bytes) 2024-09-12T09:29:43.908 INFO:teuthology.orchestra.run.smithi007.stdout:576 writesame 0x97a2165 thru 0x97b10a3 data_size 0x1(0xef3f bytes) 2024-09-12T09:29:43.931 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:43.931 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:43.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:43.991 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:43.990+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:43.992 INFO:teuthology.orchestra.run.smithi007.stdout:578 trunc from 0xeb0c418 to 0xa45b11e 2024-09-12T09:29:44.006 INFO:teuthology.orchestra.run.smithi007.stdout:579 write 0x813d9e1 thru 0x8142247 (0x4867 bytes) 2024-09-12T09:29:44.008 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:44.008 INFO:teuthology.orchestra.run.smithi007.stdout:581 read 0x3fa9b1 thru 0x3faeaa (0x4fa bytes) 2024-09-12T09:29:44.009 INFO:teuthology.orchestra.run.smithi007.stdout:582 write 0x6cbe3db thru 0x6ccb465 (0xd08b bytes) 2024-09-12T09:29:44.012 INFO:teuthology.orchestra.run.smithi007.stdout:583 write 0x3e50fc2 thru 0x3e5d68b (0xc6ca bytes) 2024-09-12T09:29:44.015 INFO:teuthology.orchestra.run.smithi007.stdout:586 write 0x81b2d0b thru 0x81c03fb (0xd6f1 bytes) 2024-09-12T09:29:44.033 INFO:teuthology.orchestra.run.smithi007.stdout:587 writesame 0xc5932b7 thru 0xc59a262 data_size 0xe(0x6fac bytes) 2024-09-12T09:29:44.036 INFO:teuthology.orchestra.run.smithi007.stdout:588 read 0x3939fbe thru 0x394717c (0xd1bf bytes) 2024-09-12T09:29:44.037 INFO:teuthology.orchestra.run.smithi007.stdout:589 write 0x951b771 thru 0x951e947 (0x31d7 bytes) 2024-09-12T09:29:44.039 INFO:teuthology.orchestra.run.smithi007.stdout:590 read 0x31a7bd2 thru 0x31b2a69 (0xae98 bytes) 2024-09-12T09:29:44.040 INFO:teuthology.orchestra.run.smithi007.stdout:591 writesame 0x81227e5 thru 0x812cc22 data_size 0x1(0xa43e bytes) 2024-09-12T09:29:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T09:29:44.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T09:29:44.075 INFO:teuthology.orchestra.run.smithi007.stdout:592 clone 3 order 18 su 8192 sc 4 2024-09-12T09:29:44.084 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:44.084 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:44.092 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:44.090+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:44.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:44.194 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:44.190+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:44.264 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:44.265 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:44.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:44.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:44.319 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:44.318+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:44.395 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:44.394+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:44.397 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:44.397 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:44.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:44.420 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:44.438 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:44.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:44.456 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:44.474 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:44.492 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:44.492 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:44.496 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:44.494+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:44.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:44.533 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:44.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:44.580 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:745 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2363250~51975] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.6609272380000002, 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001823648, 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 745 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918402+0000", 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918404+0000", 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0159999999999998e-06 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918462+0000", 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.834e-05 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918483+0000", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0208999999999999e-05 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918492+0000", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3489999999999998e-06 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918562+0000", 2024-09-12T09:29:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0344000000000003e-05 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918635+0000", 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2742000000000003e-05 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.918761+0000", 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126291 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.919854+0000", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010925830000000001 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920156+0000", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030249900000000002 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920193+0000", 2024-09-12T09:29:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6974999999999998e-05 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920226+0000", 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2299999999999999e-05 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:745 3.a e27/23 ondisk, result = 0)", 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.6592587669999999, 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000159746, 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920071+0000", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920072+0000", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920078+0000", 2024-09-12T09:29:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2700000000000001e-06 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920093+0000", 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5196e-05 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920101+0000", 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920148+0000", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6177000000000002e-05 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920153+0000", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.587e-06 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.920230+0000", 2024-09-12T09:29:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7127999999999997e-05 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:768 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.6279989610000003, 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011468400000000001, 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 768 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951330+0000", 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951332+0000", 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1009999999999999e-06 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951339+0000", 2024-09-12T09:29:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1789999999999996e-06 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951355+0000", 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6075000000000001e-05 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951367+0000", 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2342e-05 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951432+0000", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5189000000000002e-05 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951513+0000", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0994000000000004e-05 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.951605+0000", 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1802e-05 2024-09-12T09:29:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952293+0000", 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068794799999999997 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952412+0000", 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118979 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952445+0000", 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.277e-05 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952477+0000", 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2461e-05 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:768 3.a e27/23 ondisk, result = 0)", 2024-09-12T09:29:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.6269865460000004, 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013823000000000001, 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952343+0000", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-07 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952348+0000", 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4190000000000004e-06 2024-09-12T09:29:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952358+0000", 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0297e-05 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952365+0000", 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3169999999999998e-06 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952404+0000", 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8986000000000002e-05 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952409+0000", 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.203e-06 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:39.952481+0000", 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2230000000000005e-05 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:817 3.10 3:0e67bedf:::rbd_data.1191192d73a0.000000000000003b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2185468~3628] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.040419+0000", 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5389107610000003, 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003136949, 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 817 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040419+0000", 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040419+0000", 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040420+0000", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9910000000000001e-06 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040430+0000", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0769999999999997e-06 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040448+0000", 2024-09-12T09:29:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8583000000000001e-05 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040457+0000", 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9830000000000002e-06 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040517+0000", 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9639999999999998e-05 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-12T09:29:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.040521+0000", 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5709999999999999e-06 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.041994+0000", 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014723889999999999 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.042061+0000", 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7282999999999999e-05 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.042190+0000", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012855700000000001 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.043025+0000", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000835646 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.043497+0000", 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047200900000000001 2024-09-12T09:29:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.043532+0000", 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4431000000000003e-05 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.043555+0000", 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3788999999999999e-05 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:834 3.10 3:0e67bedf:::rbd_data.1191192d73a0.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.106283+0000", 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4730464049999998, 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00137612, 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 834 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106283+0000", 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106283+0000", 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106285+0000", 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4480000000000001e-06 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106295+0000", 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8330000000000006e-06 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106318+0000", 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2889999999999999e-05 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106332+0000", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4119e-05 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106414+0000", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106505+0000", 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0921999999999994e-05 2024-09-12T09:29:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.106605+0000", 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9584000000000001e-05 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107379+0000", 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00077411199999999998 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107578+0000", 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019945499999999999 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107621+0000", 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2756999999999997e-05 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107659+0000", 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7741000000000003e-05 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:834 3.10 e28/23 ondisk, result = 0)", 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.107482+0000", 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4718475519999998, 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018569999999999999, 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107482+0000", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107482+0000", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107483+0000", 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.37e-06 2024-09-12T09:29:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107490+0000", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6019999999999996e-06 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107502+0000", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2136e-05 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107509+0000", 2024-09-12T09:29:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.866e-06 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107566+0000", 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7799e-05 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107573+0000", 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1700000000000002e-06 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.107667+0000", 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4757000000000001e-05 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1013 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 533440~54361] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.407319+0000", 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1720103210000001, 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032298190000000001, 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1013 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407319+0000", 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407319+0000", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407322+0000", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6259999999999999e-06 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407396+0000", 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4375999999999994e-05 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407418+0000", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2376999999999999e-05 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407429+0000", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0713999999999999e-05 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407573+0000", 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014393899999999999 2024-09-12T09:29:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.407844+0000", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027128200000000002 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.408276+0000", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00043204700000000002 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410097+0000", 2024-09-12T09:29:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001820545 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410431+0000", 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033456300000000002 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410499+0000", 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7848000000000005e-05 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410549+0000", 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9502000000000001e-05 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:1013 3.a e28/23 ondisk, result = 0)", 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.410289+0000", 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1690406260000001, 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027153099999999999, 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410289+0000", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410289+0000", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410290+0000", 2024-09-12T09:29:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.767e-06 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410299+0000", 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0170000000000002e-06 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410323+0000", 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3802999999999999e-05 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410337+0000", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4022999999999999e-05 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410413+0000", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5968000000000002e-05 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410424+0000", 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0451999999999999e-05 2024-09-12T09:29:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.410560+0000", 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136501 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1016 3.a 3:511a9873:::rbd_data.1191192d73a0.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.422604+0000", 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1567257160000004, 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022144640000000002, 2024-09-12T09:29:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1016 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422604+0000", 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422604+0000", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422606+0000", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8770000000000002e-06 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422619+0000", 2024-09-12T09:29:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2646000000000001e-05 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422654+0000", 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5299999999999997e-05 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422671+0000", 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6903000000000001e-05 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422782+0000", 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011035999999999999 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.422918+0000", 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013641900000000001 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.423097+0000", 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017886099999999999 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.423831+0000", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073404000000000004 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424743+0000", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00091231799999999996 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424787+0000", 2024-09-12T09:29:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4172999999999999e-05 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424818+0000", 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0567000000000001e-05 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:1016 3.a e28/23 ondisk, result = 0)", 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:40.424619+0000", 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1547101150000003, 2024-09-12T09:29:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020447799999999999, 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424619+0000", 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424619+0000", 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424621+0000", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424630+0000", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6729999999999998e-06 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424653+0000", 2024-09-12T09:29:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3713000000000001e-05 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424682+0000", 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8826000000000001e-05 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424735+0000", 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2386999999999998e-05 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424740+0000", 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:40.424824+0000", 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3589000000000002e-05 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:1162 3.1e e29/23 3:7c716831:::rbd_object_map.1191192d73a0.0000000000000003:head v 29'1, mlcod=0'0)", 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:41.033057+0000", 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.546272074, 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024268240000000002, 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033057+0000", 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033057+0000", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033058+0000", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033066+0000", 2024-09-12T09:29:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.96e-06 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033092+0000", 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5976e-05 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033107+0000", 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5852999999999999e-05 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033281+0000", 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017326900000000001 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.033338+0000", 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7213999999999998e-05 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.035329+0000", 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019909670000000002 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.035484+0000", 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000155072 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1213 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:41.096724+0000", 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.4826055330000001, 2024-09-12T09:29:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027694859999999998, 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1213 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096724+0000", 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096724+0000", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096727+0000", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1829999999999998e-06 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096741+0000", 2024-09-12T09:29:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4012999999999999e-05 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096795+0000", 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3675999999999999e-05 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096819+0000", 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4363999999999999e-05 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.096957+0000", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013759199999999999 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.097198+0000", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024113199999999999 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.097508+0000", 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000309952 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.098779+0000", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012709920000000001 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099308+0000", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00052895700000000004 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099409+0000", 2024-09-12T09:29:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010108899999999999 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099493+0000", 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4536000000000005e-05 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:1213 3.a e29/23 ondisk, result = 0)", 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:41.099127+0000", 2024-09-12T09:29:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.4802024249999999, 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038206200000000002, 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099127+0000", 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099127+0000", 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099128+0000", 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6300000000000001e-06 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099138+0000", 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2369999999999998e-06 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099163+0000", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5023000000000001e-05 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099182+0000", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8987e-05 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099284+0000", 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101902 2024-09-12T09:29:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099299+0000", 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.541e-05 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:41.099509+0000", 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020987299999999999 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1335 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.142046+0000", 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.4372836329999998, 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064112869999999999, 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1335 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142046+0000", 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142046+0000", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142049+0000", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.517e-06 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142065+0000", 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6232999999999999e-05 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142099+0000", 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3827e-05 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142143+0000", 2024-09-12T09:29:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3545999999999999e-05 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142282+0000", 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013957600000000001 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.142294+0000", 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1567e-05 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.145955+0000", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036606199999999998 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.146101+0000", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014617300000000001 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.146431+0000", 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000330778 2024-09-12T09:29:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.147852+0000", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001420351 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148289+0000", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000436914 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148388+0000", 2024-09-12T09:29:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8826000000000006e-05 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148457+0000", 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9358999999999998e-05 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:1335 3.a e30/23 ondisk, result = 0)", 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.148076+0000", 2024-09-12T09:29:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.431253613, 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039655400000000003, 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148076+0000", 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148076+0000", 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148077+0000", 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4130000000000001e-06 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148086+0000", 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148110+0000", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3896e-05 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148129+0000", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8859999999999999e-05 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148251+0000", 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012252300000000001 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148276+0000", 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5018999999999999e-05 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.148472+0000", 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000196159 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1407 3.10 3:087bb392:::rbd_data.1191fc712d4f.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.204157+0000", 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.375172504, 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034311279999999999, 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1407 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204157+0000", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204157+0000", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204160+0000", 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2799999999999999e-06 2024-09-12T09:29:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204344+0000", 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000183535 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204386+0000", 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.282e-05 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204409+0000", 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2286e-05 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204522+0000", 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011353699999999999 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204707+0000", 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018473900000000001 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204965+0000", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000257574 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207039+0000", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002074284 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207310+0000", 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027113799999999998 2024-09-12T09:29:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207439+0000", 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012899600000000001 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207588+0000", 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148939 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1408 3.10 3:087bb392:::rbd_data.1191fc712d4f.0000000000000004:head [zero 4194304~14680064] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.204450+0000", 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.374879516, 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035343409999999999, 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1408 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204450+0000", 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204450+0000", 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204451+0000", 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.347e-06 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204457+0000", 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.57e-06 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204470+0000", 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3756e-05 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.204491+0000", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0720000000000002e-05 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206202+0000", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001710378 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206399+0000", 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000197245 2024-09-12T09:29:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207984+0000", 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015853250000000001 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4497.0:1407 3.10 e30/23 ondisk, result = 0)", 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.206770+0000", 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.3725591420000001, 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000282353, 2024-09-12T09:29:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206770+0000", 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206770+0000", 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206772+0000", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0020000000000001e-06 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206781+0000", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7329999999999992e-06 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206811+0000", 2024-09-12T09:29:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9754999999999999e-05 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.206825+0000", 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4218e-05 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207012+0000", 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187419 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207029+0000", 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7136e-05 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207052+0000", 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3090000000000001e-05 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing_reply(3.10 epoch 30/23 rep_tid 16 lcod 30'9)", 2024-09-12T09:29:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.207654+0000", 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.371674992, 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019265499999999999, 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207654+0000", 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207654+0000", 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207656+0000", 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3549999999999999e-06 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207677+0000", 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1525000000000001e-05 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207697+0000", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0012000000000001e-05 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207713+0000", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6297999999999999e-05 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207815+0000", 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101588 2024-09-12T09:29:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.207847+0000", 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1877e-05 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:44.686 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:44.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:29:44.688 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:44.686+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:44.689 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:44.788 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:44.786+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:44.817 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:29:44.817 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:44.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:44.889 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:44.886+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:44.990 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:44.986+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:44.996 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:45.000 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:45.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:45.052 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:29:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T09:29:45.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T09:29:45.099 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:45.098+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:45.158 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone2 from 0xc59a263 (overlap 0x197e881) to 0x0 2024-09-12T09:29:45.191 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:45.190+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949870690000001, 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013226388, 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243112+0000", 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243113+0000", 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243190+0000", 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7043000000000001e-05 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243204+0000", 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3587000000000001e-05 2024-09-12T09:29:45.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243219+0000", 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4943000000000001e-05 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249826+0000", 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006606776 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.249962+0000", 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013611499999999999 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.250284+0000", 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032211799999999998 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253909+0000", 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036251310000000002 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256018+0000", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021093829999999998 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256216+0000", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019755900000000001 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256339+0000", 2024-09-12T09:29:45.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012262899999999999 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949711527000002, 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013432342, 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 45 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243271+0000", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243272+0000", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6599999999999994e-07 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243281+0000", 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8240000000000006e-06 2024-09-12T09:29:45.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243295+0000", 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.377e-05 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243303+0000", 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4360000000000002e-06 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251352+0000", 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080488069999999998 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251497+0000", 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014492199999999999 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.251707+0000", 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000209946 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256473+0000", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047662 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256494+0000", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256593+0000", 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8301999999999995e-05 2024-09-12T09:29:45.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256704+0000", 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111079 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949663023999999, 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014730705, 2024-09-12T09:29:45.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 46 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243320+0000", 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243321+0000", 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.47e-07 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243329+0000", 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8059999999999995e-06 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243338+0000", 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8400000000000007e-06 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243346+0000", 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5440000000000001e-06 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.252951+0000", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096052099999999994 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253091+0000", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013976900000000001 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253286+0000", 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019547499999999999 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256852+0000", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035660039999999998 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257859+0000", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010067539999999999 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257936+0000", 2024-09-12T09:29:45.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7156999999999993e-05 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258051+0000", 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114299 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949607933999999, 2024-09-12T09:29:45.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015106299, 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 47 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243375+0000", 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243376+0000", 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3300000000000001e-07 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243454+0000", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8106000000000001e-05 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243468+0000", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.375e-05 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243476+0000", 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6230000000000003e-06 2024-09-12T09:29:45.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.253981+0000", 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010505067999999999 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254222+0000", 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000240122 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.254684+0000", 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046238699999999998 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258069+0000", 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033854919999999999 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258205+0000", 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013541900000000001 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258377+0000", 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017185900000000001 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258481+0000", 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010464 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949489275000001, 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018308963000000001, 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 48 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.220 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243494+0000", 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243495+0000", 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9999999999999996e-07 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243501+0000", 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-06 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243518+0000", 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7073000000000001e-05 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243526+0000", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7770000000000002e-06 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.256954+0000", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013428383 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257083+0000", 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892900000000001 2024-09-12T09:29:45.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.257284+0000", 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020067500000000001 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261519+0000", 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042348539999999997 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261670+0000", 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015173899999999999 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261742+0000", 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1476000000000005e-05 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261803+0000", 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0956999999999999e-05 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949367772999999, 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018360113000000001, 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243615+0000", 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243617+0000", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.657e-06 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243629+0000", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.242e-05 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243650+0000", 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.109e-05 2024-09-12T09:29:45.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243665+0000", 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4163e-05 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258596+0000", 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014931444 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.258819+0000", 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022252799999999999 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.259193+0000", 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037449800000000002 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261650+0000", 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024574620000000001 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261818+0000", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016791500000000001 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261924+0000", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105213 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.261975+0000", 2024-09-12T09:29:45.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1722999999999999e-05 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.949235089999998, 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020862525999999999, 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243748+0000", 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243750+0000", 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.626e-06 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243917+0000", 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167193 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243947+0000", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0145999999999999e-05 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.243969+0000", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1654e-05 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260058+0000", 2024-09-12T09:29:45.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016089882999999999 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260195+0000", 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013658600000000001 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.260404+0000", 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020855900000000001 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264456+0000", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040527699999999998 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264476+0000", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9823999999999999e-05 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264547+0000", 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1116000000000001e-05 2024-09-12T09:29:45.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264610+0000", 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3169000000000005e-05 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.238 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.948978358000002, 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020895776000000001, 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244005+0000", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244006+0000", 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5770000000000001e-06 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244018+0000", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1636e-05 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244037+0000", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9360000000000001e-05 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244074+0000", 2024-09-12T09:29:45.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6303000000000001e-05 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262033+0000", 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017959040999999999 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262203+0000", 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017056300000000001 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.262414+0000", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021082800000000001 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264629+0000", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002215416 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264706+0000", 2024-09-12T09:29:45.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6667000000000003e-05 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264843+0000", 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136852 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.264900+0000", 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7533e-05 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.948835459000001, 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021340904000000001, 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244148+0000", 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244149+0000", 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5629999999999999e-06 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244161+0000", 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2106999999999999e-05 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244191+0000", 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0162999999999999e-05 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.244211+0000", 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.995e-05 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263126+0000", 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018914665000000001 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263373+0000", 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024669200000000002 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.263771+0000", 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039814700000000001 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265200+0000", 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001428728 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265241+0000", 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1350000000000002e-05 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265382+0000", 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141209 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.265488+0000", 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010632999999999999 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.329620321, 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010847973, 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.251 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 92 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863363+0000", 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863364+0000", 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863375+0000", 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1274000000000001e-05 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863399+0000", 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3243000000000001e-05 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863414+0000", 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5653000000000001e-05 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869692+0000", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006277721 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.869816+0000", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012430300000000001 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870084+0000", 2024-09-12T09:29:45.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026783200000000002 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.873917+0000", 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038328450000000001 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874082+0000", 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016557200000000001 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874142+0000", 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.931e-05 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874211+0000", 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8869999999999996e-05 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.329537310999999, 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012610355, 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 93 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863446+0000", 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863447+0000", 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4270000000000001e-06 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863601+0000", 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015352900000000001 2024-09-12T09:29:45.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863623+0000", 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2506000000000001e-05 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863639+0000", 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6347000000000001e-05 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870855+0000", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072158680000000003 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.870984+0000", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012892599999999999 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.871189+0000", 2024-09-12T09:29:45.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020435200000000001 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874344+0000", 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031557970000000001 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875801+0000", 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001456684 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.875949+0000", 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014783600000000001 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876056+0000", 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107083 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4195.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.329308926, 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014766520999999999, 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4195", 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4137722942", 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 94 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863674+0000", 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863676+0000", 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.64e-06 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863686+0000", 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0757000000000001e-05 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863706+0000", 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9343000000000001e-05 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.863721+0000", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4946999999999999e-05 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874405+0000", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010684129000000001 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874528+0000", 2024-09-12T09:29:45.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012332599999999999 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.874725+0000", 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019663600000000001 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.876165+0000", 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001440621 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878296+0000", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021301430000000001 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878380+0000", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4462999999999994e-05 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:17.878441+0000", 2024-09-12T09:29:45.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0516000000000001e-05 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1371 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.167183+0000", 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0258001719999998, 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012700477, 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1371 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167183+0000", 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167183+0000", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167186+0000", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3799999999999998e-06 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167206+0000", 2024-09-12T09:29:45.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9490000000000001e-05 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167254+0000", 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7836000000000001e-05 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167278+0000", 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4853000000000001e-05 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167447+0000", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016827899999999999 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167677+0000", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000230812 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.168333+0000", 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065547299999999995 2024-09-12T09:29:45.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179536+0000", 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011202989999999999 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179584+0000", 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8009000000000001e-05 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179742+0000", 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015798599999999999 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179883+0000", 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014136899999999999 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1373 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.167365+0000", 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0256176930000001, 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012953706000000001, 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1373 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167365+0000", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167365+0000", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167367+0000", 2024-09-12T09:29:45.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7069999999999999e-06 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167378+0000", 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1273e-05 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167398+0000", 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9327e-05 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167415+0000", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7246000000000001e-05 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169493+0000", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002077967 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169758+0000", 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000265052 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170704+0000", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094609399999999997 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179913+0000", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092090520000000006 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180070+0000", 2024-09-12T09:29:45.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015676899999999999 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180197+0000", 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000126759 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180319+0000", 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012245999999999999 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1375 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.167452+0000", 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0255305039999998, 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013261804, 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1375 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167452+0000", 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167452+0000", 2024-09-12T09:29:45.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167454+0000", 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4729999999999999e-06 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167464+0000", 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0353999999999999e-05 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167485+0000", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0245999999999999e-05 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167503+0000", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8107000000000001e-05 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.171835+0000", 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043319259999999998 2024-09-12T09:29:45.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.172082+0000", 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024749499999999997 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.173155+0000", 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010730099999999999 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.179948+0000", 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067934040000000003 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180498+0000", 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054939099999999999 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180600+0000", 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101685 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180714+0000", 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114713 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1377 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.167550+0000", 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0254330710000001, 2024-09-12T09:29:45.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013767214, 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1377 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167550+0000", 2024-09-12T09:29:45.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167550+0000", 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167551+0000", 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4899999999999999e-06 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167563+0000", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1327e-05 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167584+0000", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1195999999999998e-05 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167595+0000", 2024-09-12T09:29:45.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1503e-05 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.174423+0000", 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006827774 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.174722+0000", 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029868499999999998 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.175739+0000", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001016821 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180370+0000", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046312469999999998 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.180907+0000", 2024-09-12T09:29:45.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00053695999999999995 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.181177+0000", 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027043500000000002 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.181317+0000", 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013977600000000001 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1379 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.167612+0000", 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0253714619999998, 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021520125000000001, 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1379 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167612+0000", 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167612+0000", 2024-09-12T09:29:45.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167612+0000", 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1399999999999996e-07 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167617+0000", 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9960000000000001e-06 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167628+0000", 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0176999999999999e-05 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.167636+0000", 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1429999999999998e-06 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.177184+0000", 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0095480679999999998 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.177436+0000", 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000252021 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.178450+0000", 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001014584 2024-09-12T09:29:45.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.185947+0000", 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074967860000000001 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.188851+0000", 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029036050000000001 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189021+0000", 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016984199999999999 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189132+0000", 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011108900000000001 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1383 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.169068+0000", 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.023915401, 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020434839, 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1383 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169068+0000", 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169068+0000", 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169070+0000", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8600000000000001e-06 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169086+0000", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5324000000000001e-05 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169140+0000", 2024-09-12T09:29:45.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4048999999999997e-05 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.169161+0000", 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1189999999999999e-05 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.181472+0000", 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01231086 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.181616+0000", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014369599999999999 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.182404+0000", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078836200000000003 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189239+0000", 2024-09-12T09:29:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006835163 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189277+0000", 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8129e-05 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189398+0000", 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000120497 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189502+0000", 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104709 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1385 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.170389+0000", 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0225939749999999, 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023043032000000001, 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1385 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170389+0000", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170389+0000", 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170391+0000", 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3130000000000001e-06 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170402+0000", 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1147e-05 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170429+0000", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6041999999999999e-05 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170443+0000", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4773999999999999e-05 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.183518+0000", 2024-09-12T09:29:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013074798 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.183750+0000", 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000231752 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.184729+0000", 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00097948400000000008 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.189535+0000", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004805223 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.193033+0000", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034980749999999998 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.193299+0000", 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266622 2024-09-12T09:29:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.193432+0000", 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013280200000000001 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:1387 3.11 3:8fb49964:::rbd_object_map.1191fc712d4f.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:42.170798+0000", 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.0221851219999998, 2024-09-12T09:29:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02403808, 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1387 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170798+0000", 2024-09-12T09:29:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170798+0000", 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170800+0000", 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0839999999999999e-06 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170813+0000", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.256e-05 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170841+0000", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8146000000000001e-05 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.170864+0000", 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2847e-05 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.186065+0000", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015201485000000001 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.186224+0000", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000159195 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.187061+0000", 2024-09-12T09:29:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00083637100000000003 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.193536+0000", 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064754870000000003 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.194334+0000", 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00079780200000000002 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.194550+0000", 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021643899999999999 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:42.194836+0000", 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566399999999998 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:45.322 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:45.323 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-12T09:29:45.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:45.325 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:45.322+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:45.371 INFO:teuthology.orchestra.run.smithi007.stdout:594 trunc from 0xc59a263 to 0x8b883d7 2024-09-12T09:29:45.425 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:45.422+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:45.447 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:45.447 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:45.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:596 trunc from 0x8b883d7 to 0x31dec30 2024-09-12T09:29:45.526 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:45.522+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:45.556 INFO:teuthology.orchestra.run.smithi007.stdout:597 punch from 0x1806a2f to 0x180f2db, (0x88ac bytes) 2024-09-12T09:29:45.556 INFO:teuthology.orchestra.run.smithi007.stdout:598 flatten 2024-09-12T09:29:45.606 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:45.606 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:45.627 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:45.626+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:45.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:45.727 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:45.726+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:45.810 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:45.810 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:45.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:45.810 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2919 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 942080~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.233195025, 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041365445000000001, 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2919 2024-09-12T09:29:45.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575880+0000", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7500000000000001e-07 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575881+0000", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575884+0000", 2024-09-12T09:29:45.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4060000000000001e-06 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575887+0000", 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7930000000000002e-06 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616841+0000", 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040954162000000002 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616963+0000", 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012160800000000001 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617245+0000", 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028208299999999998 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2920 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 974848~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.23318349199999999, 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041906318999999997, 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2920 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575892+0000", 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0950000000000001e-06 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575895+0000", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575902+0000", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4489999999999996e-06 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617280+0000", 2024-09-12T09:29:45.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041377835000000002 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617372+0000", 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2070000000000004e-05 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617797+0000", 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000425284 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2921 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1007616~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.23316678900000001, 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042305425000000001, 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2921 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575909+0000", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429999999999999e-06 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575912+0000", 2024-09-12T09:29:45.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5659999999999998e-06 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575915+0000", 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8880000000000001e-06 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617854+0000", 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041939002000000003 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617933+0000", 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8893000000000005e-05 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618213+0000", 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000279671 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2922 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1040384~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.23315476900000001, 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042845688999999999, 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2922 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575921+0000", 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0139999999999999e-06 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575924+0000", 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3330000000000001e-06 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575927+0000", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6469999999999998e-06 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618438+0000", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042511372999999998 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618552+0000", 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114024 2024-09-12T09:29:45.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618765+0000", 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213069 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5493 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060339641999999999, 2024-09-12T09:29:45.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041875908000000003, 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5493 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748736+0000", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1939999999999999e-06 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748746+0000", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5969999999999998e-06 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748752+0000", 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-12T09:29:45.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790107+0000", 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041354634000000001 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790210+0000", 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010341899999999999 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790611+0000", 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000400212 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5494 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060314025, 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042226430000000002, 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5494 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748761+0000", 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748762+0000", 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.739e-06 2024-09-12T09:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748767+0000", 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1609999999999996e-06 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748770+0000", 2024-09-12T09:29:45.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9500000000000001e-06 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790647+0000", 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041877437000000003 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790753+0000", 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105726 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790987+0000", 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023401800000000001 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5495 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060289162, 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042594770999999997, 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5495 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748787+0000", 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2810000000000001e-06 2024-09-12T09:29:45.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748796+0000", 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748801+0000", 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3850000000000001e-06 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791018+0000", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042216562999999999 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791093+0000", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5080999999999995e-05 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791380+0000", 2024-09-12T09:29:45.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028708999999999999 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5496 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2744320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060268508999999998, 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042916190999999999, 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5496 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.35e-07 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748807+0000", 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.237e-06 2024-09-12T09:29:45.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748810+0000", 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2210000000000002e-06 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748813+0000", 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5959999999999998e-06 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791412+0000", 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042598534 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791488+0000", 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5994000000000005e-05 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791722+0000", 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234374 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5497 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060257333000000003, 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043287796000000003, 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5497 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:45.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748818+0000", 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.113e-06 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748828+0000", 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9019999999999994e-06 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748835+0000", 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4400000000000002e-06 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791754+0000", 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042919481000000002 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791836+0000", 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1514000000000006e-05 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792105+0000", 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000269125 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5498 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060223939999999997, 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043631883000000003, 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5498 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748851+0000", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748852+0000", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7680000000000001e-06 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748857+0000", 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5510000000000001e-06 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748860+0000", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.157e-06 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792142+0000", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043281687999999999 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792246+0000", 2024-09-12T09:29:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103749 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792482+0000", 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023665099999999999 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5499 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2842624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:45.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060209691000000003, 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043996252, 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5499 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:29:45.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748885+0000", 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0276999999999999e-05 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748894+0000", 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7809999999999994e-06 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748898+0000", 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6669999999999998e-06 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792530+0000", 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043632399000000002 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792612+0000", 2024-09-12T09:29:45.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2317999999999993e-05 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792861+0000", 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024859200000000001 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5500 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:45.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060170060999999997, 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044345271999999998, 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5500 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748905+0000", 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748906+0000", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2300000000000001e-06 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748910+0000", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2289999999999997e-06 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748913+0000", 2024-09-12T09:29:45.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7439999999999999e-06 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792921+0000", 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044007743000000002 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793009+0000", 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8284999999999999e-05 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793250+0000", 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000240673 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5501 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2908160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060157593000000002, 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044652050999999998, 2024-09-12T09:29:45.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5501 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748918+0000", 2024-09-12T09:29:45.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.128e-06 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748921+0000", 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9890000000000001e-06 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748924+0000", 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793279+0000", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044355798000000002 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793363+0000", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3100000000000001e-05 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793569+0000", 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020620799999999999 2024-09-12T09:29:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5502 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060147067999999998, 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045033795000000001, 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5502 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748929+0000", 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2610000000000001e-06 2024-09-12T09:29:45.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748934+0000", 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2340000000000002e-06 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748938+0000", 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2640000000000002e-06 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793678+0000", 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044739745999999997 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793730+0000", 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909000000000001e-05 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793961+0000", 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023114199999999999 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5503 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060129461000000002, 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045434384000000001, 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5503 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748947+0000", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.362e-06 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748951+0000", 2024-09-12T09:29:45.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5539999999999999e-06 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748954+0000", 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794016+0000", 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045062229000000002 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794103+0000", 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7168000000000003e-05 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794379+0000", 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027592900000000001 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.885 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5504 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060116248999999997, 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045942016000000002, 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5504 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:45.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1800000000000002e-07 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748960+0000", 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.779e-06 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748966+0000", 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5650000000000001e-06 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748971+0000", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794420+0000", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045449472999999997 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794513+0000", 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2631000000000001e-05 2024-09-12T09:29:45.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794900+0000", 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038728900000000003 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5505 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060083540999999997, 2024-09-12T09:29:45.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046773908000000003, 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5505 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8700000000000005e-07 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748994+0000", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749020+0000", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6032000000000001e-05 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749028+0000", 2024-09-12T09:29:45.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794934+0000", 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045905796999999998 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795020+0000", 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5636000000000005e-05 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795765+0000", 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074500200000000003 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5506 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.060034425000000002, 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047076258000000003, 2024-09-12T09:29:45.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5506 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749043+0000", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7879999999999998e-06 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749057+0000", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4277000000000001e-05 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749067+0000", 2024-09-12T09:29:45.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999999e-06 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795798+0000", 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046730657000000002 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795882+0000", 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4432999999999996e-05 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796116+0000", 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023388599999999999 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5507 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.059997321999999999, 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047404593000000002, 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5507 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749080+0000", 2024-09-12T09:29:45.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7159999999999999e-06 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749088+0000", 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0250000000000002e-06 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749094+0000", 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796153+0000", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047059813999999998 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796244+0000", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0463000000000004e-05 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796482+0000", 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0002377 2024-09-12T09:29:45.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5508 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.059973119999999998, 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047813034999999997, 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:45.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5508 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:45.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749102+0000", 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749104+0000", 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749110+0000", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749116+0000", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5889999999999999e-06 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796574+0000", 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047457634999999998 2024-09-12T09:29:45.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796658+0000", 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4040000000000002e-05 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796914+0000", 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000256238 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:45.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:45.910 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:45.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:45.912 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:45.910+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:45.947 INFO:teuthology.orchestra.run.smithi007.stdout:599 read 0x223d841 thru 0x2242546 (0x4d06 bytes) 2024-09-12T09:29:45.968 INFO:teuthology.orchestra.run.smithi007.stdout:601 write 0xdd4d3bb thru 0xdd5c5f0 (0xf236 bytes) 2024-09-12T09:29:45.978 INFO:teuthology.orchestra.run.smithi007.stdout:603 writesame 0x71dcbdd thru 0x71e0e87 data_size 0x1(0x42ab bytes) 2024-09-12T09:29:45.992 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:45.992 INFO:teuthology.orchestra.run.smithi007.stdout:606 write 0x861c62f thru 0x8629faf (0xd981 bytes) 2024-09-12T09:29:46.003 INFO:teuthology.orchestra.run.smithi007.stdout:610 write 0xc04d90e thru 0xc0518c5 (0x3fb8 bytes) 2024-09-12T09:29:46.011 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:46.010+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:46.013 INFO:teuthology.orchestra.run.smithi007.stdout:611 trunc from 0xdd5c5f1 to 0xcdfe1f3 2024-09-12T09:29:46.027 INFO:teuthology.orchestra.run.smithi007.stdout:612 write 0x25b5162 thru 0x25b6115 (0xfb4 bytes) 2024-09-12T09:29:46.033 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:9046 3.4 3:225a570c:::rbd_data.11912e4e65e9.0000000000000097:head [set-alloc-hint object_size 262144 write_size 262144,write 180224~278 in=278b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:46.033582+0000", 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0010141530000000001, 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001056628, 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9046 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033582+0000", 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033582+0000", 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033584+0000", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0839999999999999e-06 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033594+0000", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0407999999999999e-05 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033616+0000", 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2254000000000001e-05 2024-09-12T09:29:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033627+0000", 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0502999999999999e-05 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033691+0000", 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3962999999999997e-05 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033789+0000", 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8550999999999999e-05 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.033886+0000", 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6596999999999997e-05 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-12T09:29:46.038 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:46.039 INFO:teuthology.orchestra.run.smithi007.stdout:613 read 0x8c0dce4 thru 0x8c12d71 (0x508e bytes) 2024-09-12T09:29:46.039 INFO:teuthology.orchestra.run.smithi007.stdout:615 punch from 0x57e4e97 to 0x57eb52b, (0x6694 bytes) 2024-09-12T09:29:46.039 INFO:teuthology.orchestra.run.smithi007.stdout:616 write 0x5d0a446 thru 0x5d12b0a (0x86c5 bytes) 2024-09-12T09:29:46.047 INFO:teuthology.orchestra.run.smithi007.stdout:617 read 0x53c06f8 thru 0x53cf23b (0xeb44 bytes) 2024-09-12T09:29:46.047 INFO:teuthology.orchestra.run.smithi007.stdout:618 read 0x99966fb thru 0x999836a (0x1c70 bytes) 2024-09-12T09:29:46.047 INFO:teuthology.orchestra.run.smithi007.stdout:619 read 0x4e09ccc thru 0x4e0c12d (0x2462 bytes) 2024-09-12T09:29:46.047 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:46.047 INFO:teuthology.orchestra.run.smithi007.stdout:621 write 0x5eac30b thru 0x5eafac3 (0x37b9 bytes) 2024-09-12T09:29:46.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:46.053 INFO:teuthology.orchestra.run.smithi007.stdout:622 read 0x7315ec7 thru 0x731e503 (0x863d bytes) 2024-09-12T09:29:46.054 INFO:teuthology.orchestra.run.smithi007.stdout:623 writesame 0x10e8ea1 thru 0x10ea844 data_size 0x6(0x19a4 bytes) 2024-09-12T09:29:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T09:29:46.065 INFO:teuthology.orchestra.run.smithi007.stdout:624 read 0x699e7a6 thru 0x69abee6 (0xd741 bytes) 2024-09-12T09:29:46.065 INFO:teuthology.orchestra.run.smithi007.stdout:625 write 0x45776eb thru 0x45860de (0xe9f4 bytes) 2024-09-12T09:29:46.088 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T09:29:46.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:46.090 INFO:teuthology.orchestra.run.smithi007.stdout:626 write 0xe202f8f thru 0xe21278c (0xf7fe bytes) 2024-09-12T09:29:46.100 INFO:teuthology.orchestra.run.smithi007.stdout:627 write 0x447a2fb thru 0x4482df9 (0x8aff bytes) 2024-09-12T09:29:46.107 INFO:teuthology.orchestra.run.smithi007.stdout:628 writesame 0xbefeabc thru 0xbeffe6b data_size 0x1e(0x13b0 bytes) 2024-09-12T09:29:46.112 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:46.110+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:46.113 INFO:teuthology.orchestra.run.smithi007.stdout:630 punch from 0x118be17 to 0x118d9c5, (0x1bae bytes) 2024-09-12T09:29:46.113 INFO:teuthology.orchestra.run.smithi007.stdout:631 writesame 0x215611 thru 0x219725 data_size 0x1(0x4115 bytes) 2024-09-12T09:29:46.119 INFO:teuthology.orchestra.run.smithi007.stdout:632 writesame 0x2ec058e thru 0x2ec8c37 data_size 0xb(0x86aa bytes) 2024-09-12T09:29:46.129 INFO:teuthology.orchestra.run.smithi007.stdout:633 read 0xcd4c959 thru 0xcd4d3ec (0xa94 bytes) 2024-09-12T09:29:46.129 INFO:teuthology.orchestra.run.smithi007.stdout:634 punch from 0xd5a7dfb to 0xd5b47ea, (0xc9ef bytes) 2024-09-12T09:29:46.129 INFO:teuthology.orchestra.run.smithi007.stdout:636 writesame 0x7808d99 thru 0x781772f data_size 0x1f(0xe997 bytes) 2024-09-12T09:29:46.138 INFO:teuthology.orchestra.run.smithi007.stdout:638 punch from 0xdce91f7 to 0xdcf16b9, (0x84c2 bytes) 2024-09-12T09:29:46.138 INFO:teuthology.orchestra.run.smithi007.stdout:640 read 0x22b0c24 thru 0x22c09a7 (0xfd84 bytes) 2024-09-12T09:29:46.138 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:46.138 INFO:teuthology.orchestra.run.smithi007.stdout:648 write 0x489bd7e thru 0x48a3018 (0x729b bytes) 2024-09-12T09:29:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:650 trunc from 0xe21278d to 0x5f2eca7 2024-09-12T09:29:46.176 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:46.177 INFO:teuthology.orchestra.run.smithi007.stdout:652 read 0x5e54127 thru 0x5e5d0a5 (0x8f7f bytes) 2024-09-12T09:29:46.177 INFO:teuthology.orchestra.run.smithi007.stdout:653 write 0x4d1164d thru 0x4d1fe36 (0xe7ea bytes) 2024-09-12T09:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:46.184 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:46.190 INFO:teuthology.orchestra.run.smithi007.stdout:654 read 0x36e7985 thru 0x36eae86 (0x3502 bytes) 2024-09-12T09:29:46.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:46.212 INFO:teuthology.orchestra.run.smithi007.stdout:655 write 0xe5afd87 thru 0xe5beca7 (0xef21 bytes) 2024-09-12T09:29:46.212 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:46.210+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:46.234 INFO:teuthology.orchestra.run.smithi007.stdout:657 trunc from 0xe5beca8 to 0x576c0da 2024-09-12T09:29:46.255 INFO:teuthology.orchestra.run.smithi007.stdout:658 read 0xd8fb99 thru 0xd9f8fd (0xfd65 bytes) 2024-09-12T09:29:46.274 INFO:teuthology.orchestra.run.smithi007.stdout:659 write 0xb95fa19 thru 0xb9618a3 (0x1e8b bytes) 2024-09-12T09:29:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:662 read 0xac70e24 thru 0xac777e6 (0x69c3 bytes) 2024-09-12T09:29:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:664 punch from 0x18d7426 to 0x18e3d51, (0xc92b bytes) 2024-09-12T09:29:46.294 INFO:teuthology.orchestra.run.smithi007.stdout:668 write 0xe09a8af thru 0xe09aeae (0x600 bytes) 2024-09-12T09:29:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:669 clone 4 order 25 su 16777216 sc 9 2024-09-12T09:29:46.313 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:46.310+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:46.341 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2895616429999999, 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:29:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56900985900000001, 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56750586700000005, 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:29:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56748311799999995, 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:29:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56745967799999997, 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:29:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56743622999999999, 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:29:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:29:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56739279399999998, 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.56735599699999995, 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:29:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:29:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49936775700000002, 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:29:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:29:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49931219700000001, 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:29:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.499282948, 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:29:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49925924399999999, 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:29:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49923951599999999, 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:29:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:29:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:29:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.499215994, 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49919301999999999, 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:29:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:29:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49914883100000002, 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:29:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49911446399999998, 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:29:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.44569109499999998, 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:29:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8464 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5201920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.445136802, 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016641282, 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8464 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894360+0000", 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:29:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894364+0000", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0879999999999999e-06 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894366+0000", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.469e-06 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.910874+0000", 2024-09-12T09:29:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016508228999999999 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.910955+0000", 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1094000000000007e-05 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911000+0000", 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4660000000000003e-05 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.44512625900000002, 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:29:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:46.436 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:46.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:46.441 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-12T09:29:46.063817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77448547,"num_objects":57,"num_object_clones":4,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":2,"num_read":5089,"num_read_kb":13969,"num_write":846,"num_write_kb":27024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253396,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721324,"statfs":{"total":773094113280,"available":772834635776,"internally_reserved":0,"allocated":3956736,"data_stored":6961324,"data_compressed":512837,"data_compressed_allocated":880640,"data_compressed_original":5697536,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":66520438,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":2,"num_read":2255,"num_read_kb":11465,"num_write":165,"num_write_kb":23346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030393"},"pg_stats":[{"pgid":"3.1f","version":"29'239","reported_seq":270,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.031302+0000","last_change":"2024-09-12T09:29:36.694097+0000","last_active":"2024-09-12T09:29:41.031302+0000","last_peered":"2024-09-12T09:29:41.031302+0000","last_clean":"2024-09-12T09:29:41.031302+0000","last_became_active":"2024-09-12T09:29:34.667276+0000","last_became_peered":"2024-09-12T09:29:34.667276+0000","last_unstale":"2024-09-12T09:29:41.031302+0000","last_undegraded":"2024-09-12T09:29:41.031302+0000","last_fullsized":"2024-09-12T09:29:41.031302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:46.977468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":65,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1117,"num_read_kb":880,"num_write":378,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"33'22","reported_seq":708,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:46.018716+0000","last_change":"2024-09-12T09:29:36.673655+0000","last_active":"2024-09-12T09:29:46.018716+0000","last_peered":"2024-09-12T09:29:46.018716+0000","last_clean":"2024-09-12T09:29:46.018716+0000","last_became_active":"2024-09-12T09:29:34.661193+0000","last_became_peered":"2024-09-12T09:29:34.661193+0000","last_unstale":"2024-09-12T09:29:46.018716+0000","last_undegraded":"2024-09-12T09:29:46.018716+0000","last_fullsized":"2024-09-12T09:29:46.018716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:25.653734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":17145905,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":674,"num_read_kb":5164,"num_write":23,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"28'21","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.067004+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:29:45.067004+0000","last_peered":"2024-09-12T09:29:45.067004+0000","last_clean":"2024-09-12T09:29:45.067004+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:29:45.067004+0000","last_undegraded":"2024-09-12T09:29:45.067004+0000","last_fullsized":"2024-09-12T09:29:45.067004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:53.271647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"33'5","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.274056+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:29:45.274056+0000","last_peered":"2024-09-12T09:29:45.274056+0000","last_clean":"2024-09-12T09:29:45.274056+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:29:45.274056+0000","last_undegraded":"2024-09-12T09:29:45.274056+0000","last_fullsized":"2024-09-12T09:29:45.274056+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:29:22.848981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":6,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"28'5","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030682+0000","last_change":"2024-09-12T09:29:36.693564+0000","last_active":"2024-09-12T09:29:41.030682+0000","last_peered":"2024-09-12T09:29:41.030682+0000","last_clean":"2024-09-12T09:29:41.030682+0000","last_became_active":"2024-09-12T09:29:34.673730+0000","last_became_peered":"2024-09-12T09:29:34.673730+0000","last_unstale":"2024-09-12T09:29:41.030682+0000","last_undegraded":"2024-09-12T09:29:41.030682+0000","last_fullsized":"2024-09-12T09:29:41.030682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:14.194138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":3724517,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":54,"num_write":5,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"26'2","reported_seq":133,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051025+0000","last_change":"2024-09-12T09:29:36.673515+0000","last_active":"2024-09-12T09:29:42.051025+0000","last_peered":"2024-09-12T09:29:42.051025+0000","last_clean":"2024-09-12T09:29:42.051025+0000","last_became_active":"2024-09-12T09:29:34.665723+0000","last_became_peered":"2024-09-12T09:29:34.665723+0000","last_unstale":"2024-09-12T09:29:42.051025+0000","last_undegraded":"2024-09-12T09:29:42.051025+0000","last_fullsized":"2024-09-12T09:29:42.051025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:29.913628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"28'8","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020184+0000","last_change":"2024-09-12T09:29:36.680185+0000","last_active":"2024-09-12T09:29:41.020184+0000","last_peered":"2024-09-12T09:29:41.020184+0000","last_clean":"2024-09-12T09:29:41.020184+0000","last_became_active":"2024-09-12T09:29:34.667496+0000","last_became_peered":"2024-09-12T09:29:34.667496+0000","last_unstale":"2024-09-12T09:29:41.020184+0000","last_undegraded":"2024-09-12T09:29:41.020184+0000","last_fullsized":"2024-09-12T09:29:41.020184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:32:30.033634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":3655626,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"33'9","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.791928+0000","last_change":"2024-09-12T09:29:36.672431+0000","last_active":"2024-09-12T09:29:45.791928+0000","last_peered":"2024-09-12T09:29:45.791928+0000","last_clean":"2024-09-12T09:29:45.791928+0000","last_became_active":"2024-09-12T09:29:34.674564+0000","last_became_peered":"2024-09-12T09:29:34.674564+0000","last_unstale":"2024-09-12T09:29:45.791928+0000","last_undegraded":"2024-09-12T09:29:45.791928+0000","last_fullsized":"2024-09-12T09:29:45.791928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:06.078195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":495240,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"28'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030944+0000","last_change":"2024-09-12T09:29:36.694115+0000","last_active":"2024-09-12T09:29:41.030944+0000","last_peered":"2024-09-12T09:29:41.030944+0000","last_clean":"2024-09-12T09:29:41.030944+0000","last_became_active":"2024-09-12T09:29:34.667951+0000","last_became_peered":"2024-09-12T09:29:34.667951+0000","last_unstale":"2024-09-12T09:29:41.030944+0000","last_undegraded":"2024-09-12T09:29:41.030944+0000","last_fullsized":"2024-09-12T09:29:41.030944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:46.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"33'12","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:46.011440+0000","last_change":"2024-09-12T09:29:36.686819+0000","last_active":"2024-09-12T09:29:46.011440+0000","last_peered":"2024-09-12T09:29:46.011440+0000","last_clean":"2024-09-12T09:29:46.011440+0000","last_became_active":"2024-09-12T09:29:34.675045+0000","last_became_peered":"2024-09-12T09:29:34.675045+0000","last_unstale":"2024-09-12T09:29:46.011440+0000","last_undegraded":"2024-09-12T09:29:46.011440+0000","last_fullsized":"2024-09-12T09:29:46.011440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:11.513987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":3288812,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":80,"num_write":11,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037142+0000","last_change":"2024-09-12T09:29:36.762197+0000","last_active":"2024-09-12T09:29:42.037142+0000","last_peered":"2024-09-12T09:29:42.037142+0000","last_clean":"2024-09-12T09:29:42.037142+0000","last_became_active":"2024-09-12T09:29:34.683802+0000","last_became_peered":"2024-09-12T09:29:34.683802+0000","last_unstale":"2024-09-12T09:29:42.037142+0000","last_undegraded":"2024-09-12T09:29:42.037142+0000","last_fullsized":"2024-09-12T09:29:42.037142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:55.340966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"28'8","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051065+0000","last_change":"2024-09-12T09:29:36.674037+0000","last_active":"2024-09-12T09:29:42.051065+0000","last_peered":"2024-09-12T09:29:42.051065+0000","last_clean":"2024-09-12T09:29:42.051065+0000","last_became_active":"2024-09-12T09:29:34.670141+0000","last_became_peered":"2024-09-12T09:29:34.670141+0000","last_unstale":"2024-09-12T09:29:42.051065+0000","last_undegraded":"2024-09-12T09:29:42.051065+0000","last_fullsized":"2024-09-12T09:29:42.051065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:40:07.866266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":2231814,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.147589+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:45.147589+0000","last_peered":"2024-09-12T09:29:45.147589+0000","last_clean":"2024-09-12T09:29:45.147589+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:45.147589+0000","last_undegraded":"2024-09-12T09:29:45.147589+0000","last_fullsized":"2024-09-12T09:29:45.147589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030748+0000","last_change":"2024-09-12T09:29:36.693597+0000","last_active":"2024-09-12T09:29:41.030748+0000","last_peered":"2024-09-12T09:29:41.030748+0000","last_clean":"2024-09-12T09:29:41.030748+0000","last_became_active":"2024-09-12T09:29:34.665541+0000","last_became_peered":"2024-09-12T09:29:34.665541+0000","last_unstale":"2024-09-12T09:29:41.030748+0000","last_undegraded":"2024-09-12T09:29:41.030748+0000","last_fullsized":"2024-09-12T09:29:41.030748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:05.150134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020258+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:41.020258+0000","last_peered":"2024-09-12T09:29:41.020258+0000","last_clean":"2024-09-12T09:29:41.020258+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:41.020258+0000","last_undegraded":"2024-09-12T09:29:41.020258+0000","last_fullsized":"2024-09-12T09:29:41.020258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"33'6","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.505852+0000","last_change":"2024-09-12T09:29:36.673901+0000","last_active":"2024-09-12T09:29:45.505852+0000","last_peered":"2024-09-12T09:29:45.505852+0000","last_clean":"2024-09-12T09:29:45.505852+0000","last_became_active":"2024-09-12T09:29:34.670228+0000","last_became_peered":"2024-09-12T09:29:34.670228+0000","last_unstale":"2024-09-12T09:29:45.505852+0000","last_undegraded":"2024-09-12T09:29:45.505852+0000","last_fullsized":"2024-09-12T09:29:45.505852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:42.075722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":278448,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037630+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:42.037630+0000","last_peered":"2024-09-12T09:29:42.037630+0000","last_clean":"2024-09-12T09:29:42.037630+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:42.037630+0000","last_undegraded":"2024-09-12T09:29:42.037630+0000","last_fullsized":"2024-09-12T09:29:42.037630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020839+0000","last_change":"2024-09-12T09:29:36.680262+0000","last_active":"2024-09-12T09:29:41.020839+0000","last_peered":"2024-09-12T09:29:41.020839+0000","last_clean":"2024-09-12T09:29:41.020839+0000","last_became_active":"2024-09-12T09:29:34.667442+0000","last_became_peered":"2024-09-12T09:29:34.667442+0000","last_unstale":"2024-09-12T09:29:41.020839+0000","last_undegraded":"2024-09-12T09:29:41.020839+0000","last_fullsized":"2024-09-12T09:29:41.020839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:52.348124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.021072+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:41.021072+0000","last_peered":"2024-09-12T09:29:41.021072+0000","last_clean":"2024-09-12T09:29:41.021072+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:41.021072+0000","last_undegraded":"2024-09-12T09:29:41.021072+0000","last_fullsized":"2024-09-12T09:29:41.021072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"33'19","reported_seq":172,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.326721+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:29:45.326721+0000","last_peered":"2024-09-12T09:29:45.326721+0000","last_clean":"2024-09-12T09:29:45.326721+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:29:45.326721+0000","last_undegraded":"2024-09-12T09:29:45.326721+0000","last_fullsized":"2024-09-12T09:29:45.326721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:23.342932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":2915227,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":354,"num_read_kb":365,"num_write":17,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'7","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051616+0000","last_change":"2024-09-12T09:29:36.673934+0000","last_active":"2024-09-12T09:29:42.051616+0000","last_peered":"2024-09-12T09:29:42.051616+0000","last_clean":"2024-09-12T09:29:42.051616+0000","last_became_active":"2024-09-12T09:29:34.669115+0000","last_became_peered":"2024-09-12T09:29:34.669115+0000","last_unstale":"2024-09-12T09:29:42.051616+0000","last_undegraded":"2024-09-12T09:29:42.051616+0000","last_fullsized":"2024-09-12T09:29:42.051616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:27.794886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":8,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.066478+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:45.066478+0000","last_peered":"2024-09-12T09:29:45.066478+0000","last_clean":"2024-09-12T09:29:45.066478+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:45.066478+0000","last_undegraded":"2024-09-12T09:29:45.066478+0000","last_fullsized":"2024-09-12T09:29:45.066478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"28'15","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.021094+0000","last_change":"2024-09-12T09:29:36.679214+0000","last_active":"2024-09-12T09:29:41.021094+0000","last_peered":"2024-09-12T09:29:41.021094+0000","last_clean":"2024-09-12T09:29:41.021094+0000","last_became_active":"2024-09-12T09:29:34.667231+0000","last_became_peered":"2024-09-12T09:29:34.667231+0000","last_unstale":"2024-09-12T09:29:41.021094+0000","last_undegraded":"2024-09-12T09:29:41.021094+0000","last_fullsized":"2024-09-12T09:29:41.021094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:23.576171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.057435+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:45.057435+0000","last_peered":"2024-09-12T09:29:45.057435+0000","last_clean":"2024-09-12T09:29:45.057435+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:45.057435+0000","last_undegraded":"2024-09-12T09:29:45.057435+0000","last_fullsized":"2024-09-12T09:29:45.057435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030418+0000","last_change":"2024-09-12T09:29:36.691601+0000","last_active":"2024-09-12T09:29:41.030418+0000","last_peered":"2024-09-12T09:29:41.030418+0000","last_clean":"2024-09-12T09:29:41.030418+0000","last_became_active":"2024-09-12T09:29:34.672667+0000","last_became_peered":"2024-09-12T09:29:34.672667+0000","last_unstale":"2024-09-12T09:29:41.030418+0000","last_undegraded":"2024-09-12T09:29:41.030418+0000","last_fullsized":"2024-09-12T09:29:41.030418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:53:57.721259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036530+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:42.036530+0000","last_peered":"2024-09-12T09:29:42.036530+0000","last_clean":"2024-09-12T09:29:42.036530+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:42.036530+0000","last_undegraded":"2024-09-12T09:29:42.036530+0000","last_fullsized":"2024-09-12T09:29:42.036530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"33'2","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.258337+0000","last_change":"2024-09-12T09:29:36.673018+0000","last_active":"2024-09-12T09:29:45.258337+0000","last_peered":"2024-09-12T09:29:45.258337+0000","last_clean":"2024-09-12T09:29:45.258337+0000","last_became_active":"2024-09-12T09:29:34.656612+0000","last_became_peered":"2024-09-12T09:29:34.656612+0000","last_unstale":"2024-09-12T09:29:45.258337+0000","last_undegraded":"2024-09-12T09:29:45.258337+0000","last_fullsized":"2024-09-12T09:29:45.258337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:32:26.402801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":262160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":3,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.066652+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:45.066652+0000","last_peered":"2024-09-12T09:29:45.066652+0000","last_clean":"2024-09-12T09:29:45.066652+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:45.066652+0000","last_undegraded":"2024-09-12T09:29:45.066652+0000","last_fullsized":"2024-09-12T09:29:45.066652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":129,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036566+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:42.036566+0000","last_peered":"2024-09-12T09:29:42.036566+0000","last_clean":"2024-09-12T09:29:42.036566+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:42.036566+0000","last_undegraded":"2024-09-12T09:29:42.036566+0000","last_fullsized":"2024-09-12T09:29:42.036566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"33'121","reported_seq":814,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.974624+0000","last_change":"2024-09-12T09:29:36.673435+0000","last_active":"2024-09-12T09:29:45.974624+0000","last_peered":"2024-09-12T09:29:45.974624+0000","last_clean":"2024-09-12T09:29:45.974624+0000","last_became_active":"2024-09-12T09:29:34.665426+0000","last_became_peered":"2024-09-12T09:29:34.665426+0000","last_unstale":"2024-09-12T09:29:45.974624+0000","last_undegraded":"2024-09-12T09:29:45.974624+0000","last_fullsized":"2024-09-12T09:29:45.974624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.584108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1938,"num_read_kb":1705,"num_write":118,"num_write_kb":543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.050755+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:29:42.050755+0000","last_peered":"2024-09-12T09:29:42.050755+0000","last_clean":"2024-09-12T09:29:42.050755+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:29:42.050755+0000","last_undegraded":"2024-09-12T09:29:42.050755+0000","last_fullsized":"2024-09-12T09:29:42.050755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:31.590981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":4115751,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"33'12","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.994576+0000","last_change":"2024-09-12T09:29:36.727472+0000","last_active":"2024-09-12T09:29:45.994576+0000","last_peered":"2024-09-12T09:29:45.994576+0000","last_clean":"2024-09-12T09:29:45.994576+0000","last_became_active":"2024-09-12T09:29:34.662340+0000","last_became_peered":"2024-09-12T09:29:34.662340+0000","last_unstale":"2024-09-12T09:29:45.994576+0000","last_undegraded":"2024-09-12T09:29:45.994576+0000","last_fullsized":"2024-09-12T09:29:45.994576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:53.890918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":417792,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":12,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"33'13","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.643765+0000","last_change":"2024-09-12T09:29:36.674453+0000","last_active":"2024-09-12T09:29:45.643765+0000","last_peered":"2024-09-12T09:29:45.643765+0000","last_clean":"2024-09-12T09:29:45.643765+0000","last_became_active":"2024-09-12T09:29:34.671250+0000","last_became_peered":"2024-09-12T09:29:34.671250+0000","last_unstale":"2024-09-12T09:29:45.643765+0000","last_undegraded":"2024-09-12T09:29:45.643765+0000","last_fullsized":"2024-09-12T09:29:45.643765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:28:04.510986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":19520705,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":144,"num_write":11,"num_write_kb":653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"33'3","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.485461+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:29:45.485461+0000","last_peered":"2024-09-12T09:29:45.485461+0000","last_clean":"2024-09-12T09:29:45.485461+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:29:45.485461+0000","last_undegraded":"2024-09-12T09:29:45.485461+0000","last_fullsized":"2024-09-12T09:29:45.485461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:24.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037055+0000","last_change":"2024-09-12T09:29:36.763498+0000","last_active":"2024-09-12T09:29:42.037055+0000","last_peered":"2024-09-12T09:29:42.037055+0000","last_clean":"2024-09-12T09:29:42.037055+0000","last_became_active":"2024-09-12T09:29:34.678236+0000","last_became_peered":"2024-09-12T09:29:34.678236+0000","last_unstale":"2024-09-12T09:29:42.037055+0000","last_undegraded":"2024-09-12T09:29:42.037055+0000","last_fullsized":"2024-09-12T09:29:42.037055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:34:56.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037081+0000","last_change":"2024-09-12T09:29:36.762465+0000","last_active":"2024-09-12T09:29:42.037081+0000","last_peered":"2024-09-12T09:29:42.037081+0000","last_clean":"2024-09-12T09:29:42.037081+0000","last_became_active":"2024-09-12T09:29:34.681589+0000","last_became_peered":"2024-09-12T09:29:34.681589+0000","last_unstale":"2024-09-12T09:29:42.037081+0000","last_undegraded":"2024-09-12T09:29:42.037081+0000","last_fullsized":"2024-09-12T09:29:42.037081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:23.162912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"28'9","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037301+0000","last_change":"2024-09-12T09:29:36.763375+0000","last_active":"2024-09-12T09:29:42.037301+0000","last_peered":"2024-09-12T09:29:42.037301+0000","last_clean":"2024-09-12T09:29:42.037301+0000","last_became_active":"2024-09-12T09:29:34.672883+0000","last_became_peered":"2024-09-12T09:29:34.672883+0000","last_unstale":"2024-09-12T09:29:42.037301+0000","last_undegraded":"2024-09-12T09:29:42.037301+0000","last_fullsized":"2024-09-12T09:29:42.037301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:50:45.246121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"33'32","reported_seq":732,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.923442+0000","last_change":"2024-09-12T09:29:36.727208+0000","last_active":"2024-09-12T09:29:45.923442+0000","last_peered":"2024-09-12T09:29:45.923442+0000","last_clean":"2024-09-12T09:29:45.923442+0000","last_became_active":"2024-09-12T09:29:34.674419+0000","last_became_peered":"2024-09-12T09:29:34.674419+0000","last_unstale":"2024-09-12T09:29:45.923442+0000","last_undegraded":"2024-09-12T09:29:45.923442+0000","last_fullsized":"2024-09-12T09:29:45.923442+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:48:21.450188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":18199795,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":686,"num_read_kb":5381,"num_write":32,"num_write_kb":10788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'1","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037646+0000","last_change":"2024-09-12T09:29:36.761814+0000","last_active":"2024-09-12T09:29:42.037646+0000","last_peered":"2024-09-12T09:29:42.037646+0000","last_clean":"2024-09-12T09:29:42.037646+0000","last_became_active":"2024-09-12T09:29:34.675375+0000","last_became_peered":"2024-09-12T09:29:34.675375+0000","last_unstale":"2024-09-12T09:29:42.037646+0000","last_undegraded":"2024-09-12T09:29:42.037646+0000","last_fullsized":"2024-09-12T09:29:42.037646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:13.413928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030422+0000","last_change":"2024-09-12T09:29:36.691565+0000","last_active":"2024-09-12T09:29:41.030422+0000","last_peered":"2024-09-12T09:29:41.030422+0000","last_clean":"2024-09-12T09:29:41.030422+0000","last_became_active":"2024-09-12T09:29:34.670997+0000","last_became_peered":"2024-09-12T09:29:34.670997+0000","last_unstale":"2024-09-12T09:29:41.030422+0000","last_undegraded":"2024-09-12T09:29:41.030422+0000","last_fullsized":"2024-09-12T09:29:41.030422+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:21.434542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"28'9","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036838+0000","last_change":"2024-09-12T09:29:36.762929+0000","last_active":"2024-09-12T09:29:42.036838+0000","last_peered":"2024-09-12T09:29:42.036838+0000","last_clean":"2024-09-12T09:29:42.036838+0000","last_became_active":"2024-09-12T09:29:34.673272+0000","last_became_peered":"2024-09-12T09:29:34.673272+0000","last_unstale":"2024-09-12T09:29:42.036838+0000","last_undegraded":"2024-09-12T09:29:42.036838+0000","last_fullsized":"2024-09-12T09:29:42.036838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:20.910614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":76858160,"num_objects":53,"num_object_clones":4,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":2,"num_read":5023,"num_read_kb":13914,"num_write":743,"num_write_kb":25394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":9658147,"data_compressed":486276,"data_compressed_allocated":1089536,"data_compressed_original":9416704,"omap_allocated":0,"internal_metadata":0},"log_size":608,"ondisk_log_size":608,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":820048,"data_compressed":54440,"data_compressed_allocated":94208,"data_compressed_original":667648,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":450560,"data_stored":314538,"data_compressed":81867,"data_compressed_allocated":94208,"data_compressed_original":167936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":1375542,"data_compressed":77271,"data_compressed_allocated":151552,"data_compressed_original":1212416,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":573440,"data_stored":2359293,"data_compressed":1268,"data_compressed_allocated":143360,"data_compressed_original":2187264,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":684032,"data_stored":537966,"data_compressed":184161,"data_compressed_allocated":208896,"data_compressed_original":335872,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":442368,"data_stored":498439,"data_compressed":36227,"data_compressed_allocated":57344,"data_compressed_original":339968,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":164206,"data_compressed":120,"data_compressed_allocated":8192,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35820,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336020,"statfs":{"total":96636764160,"available":96600084480,"internally_reserved":0,"allocated":487424,"data_stored":891292,"data_compressed":77483,"data_compressed_allocated":122880,"data_compressed_original":745472,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":25442,"data_compressed":12808,"data_compressed_allocated":16384,"data_compressed_original":20480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":51079,"data_compressed":120,"data_compressed_allocated":8192,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":1666298,"data_compressed":37086,"data_compressed_allocated":139264,"data_compressed_original":1650688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":2797464,"data_compressed":216178,"data_compressed_allocated":380928,"data_compressed_original":2707456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":195321,"data_compressed":81867,"data_compressed_allocated":94208,"data_compressed_original":167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":3289215,"data_compressed":12161,"data_compressed_allocated":221184,"data_compressed_original":3252224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":1522865,"data_compressed":77455,"data_compressed_allocated":172032,"data_compressed_original":1482752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":110463,"data_compressed":48601,"data_compressed_allocated":57344,"data_compressed_original":94208,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:46.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:29:46.443 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:46.434+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:46.538 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:46.534+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:46.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:46.608 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:46.608 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:46.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:46.671 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:46.670+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:46.719 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:46.719 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:46.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:46.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:46.740 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:46.738+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:46.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:46.769 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:46.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:29:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:29:46.795 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-12T09:29:46.063817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77448547,"num_objects":57,"num_object_clones":4,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":2,"num_read":5089,"num_read_kb":13969,"num_write":846,"num_write_kb":27024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253396,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721324,"statfs":{"total":773094113280,"available":772834635776,"internally_reserved":0,"allocated":3956736,"data_stored":6961324,"data_compressed":512837,"data_compressed_allocated":880640,"data_compressed_original":5697536,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":66520438,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":2,"num_read":2255,"num_read_kb":11465,"num_write":165,"num_write_kb":23346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030393"},"pg_stats":[{"pgid":"3.1f","version":"29'239","reported_seq":270,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.031302+0000","last_change":"2024-09-12T09:29:36.694097+0000","last_active":"2024-09-12T09:29:41.031302+0000","last_peered":"2024-09-12T09:29:41.031302+0000","last_clean":"2024-09-12T09:29:41.031302+0000","last_became_active":"2024-09-12T09:29:34.667276+0000","last_became_peered":"2024-09-12T09:29:34.667276+0000","last_unstale":"2024-09-12T09:29:41.031302+0000","last_undegraded":"2024-09-12T09:29:41.031302+0000","last_fullsized":"2024-09-12T09:29:41.031302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:46.977468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":65,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1117,"num_read_kb":880,"num_write":378,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"33'22","reported_seq":708,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:46.018716+0000","last_change":"2024-09-12T09:29:36.673655+0000","last_active":"2024-09-12T09:29:46.018716+0000","last_peered":"2024-09-12T09:29:46.018716+0000","last_clean":"2024-09-12T09:29:46.018716+0000","last_became_active":"2024-09-12T09:29:34.661193+0000","last_became_peered":"2024-09-12T09:29:34.661193+0000","last_unstale":"2024-09-12T09:29:46.018716+0000","last_undegraded":"2024-09-12T09:29:46.018716+0000","last_fullsized":"2024-09-12T09:29:46.018716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:25.653734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":17145905,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":674,"num_read_kb":5164,"num_write":23,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"28'21","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.067004+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:29:45.067004+0000","last_peered":"2024-09-12T09:29:45.067004+0000","last_clean":"2024-09-12T09:29:45.067004+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:29:45.067004+0000","last_undegraded":"2024-09-12T09:29:45.067004+0000","last_fullsized":"2024-09-12T09:29:45.067004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:53.271647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"33'5","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.274056+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:29:45.274056+0000","last_peered":"2024-09-12T09:29:45.274056+0000","last_clean":"2024-09-12T09:29:45.274056+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:29:45.274056+0000","last_undegraded":"2024-09-12T09:29:45.274056+0000","last_fullsized":"2024-09-12T09:29:45.274056+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:29:22.848981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":6,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"28'5","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030682+0000","last_change":"2024-09-12T09:29:36.693564+0000","last_active":"2024-09-12T09:29:41.030682+0000","last_peered":"2024-09-12T09:29:41.030682+0000","last_clean":"2024-09-12T09:29:41.030682+0000","last_became_active":"2024-09-12T09:29:34.673730+0000","last_became_peered":"2024-09-12T09:29:34.673730+0000","last_unstale":"2024-09-12T09:29:41.030682+0000","last_undegraded":"2024-09-12T09:29:41.030682+0000","last_fullsized":"2024-09-12T09:29:41.030682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:14.194138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":3724517,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":54,"num_write":5,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"26'2","reported_seq":133,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051025+0000","last_change":"2024-09-12T09:29:36.673515+0000","last_active":"2024-09-12T09:29:42.051025+0000","last_peered":"2024-09-12T09:29:42.051025+0000","last_clean":"2024-09-12T09:29:42.051025+0000","last_became_active":"2024-09-12T09:29:34.665723+0000","last_became_peered":"2024-09-12T09:29:34.665723+0000","last_unstale":"2024-09-12T09:29:42.051025+0000","last_undegraded":"2024-09-12T09:29:42.051025+0000","last_fullsized":"2024-09-12T09:29:42.051025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:29.913628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"28'8","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020184+0000","last_change":"2024-09-12T09:29:36.680185+0000","last_active":"2024-09-12T09:29:41.020184+0000","last_peered":"2024-09-12T09:29:41.020184+0000","last_clean":"2024-09-12T09:29:41.020184+0000","last_became_active":"2024-09-12T09:29:34.667496+0000","last_became_peered":"2024-09-12T09:29:34.667496+0000","last_unstale":"2024-09-12T09:29:41.020184+0000","last_undegraded":"2024-09-12T09:29:41.020184+0000","last_fullsized":"2024-09-12T09:29:41.020184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:32:30.033634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":3655626,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"33'9","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.791928+0000","last_change":"2024-09-12T09:29:36.672431+0000","last_active":"2024-09-12T09:29:45.791928+0000","last_peered":"2024-09-12T09:29:45.791928+0000","last_clean":"2024-09-12T09:29:45.791928+0000","last_became_active":"2024-09-12T09:29:34.674564+0000","last_became_peered":"2024-09-12T09:29:34.674564+0000","last_unstale":"2024-09-12T09:29:45.791928+0000","last_undegraded":"2024-09-12T09:29:45.791928+0000","last_fullsized":"2024-09-12T09:29:45.791928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:06.078195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":495240,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"28'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030944+0000","last_change":"2024-09-12T09:29:36.694115+0000","last_active":"2024-09-12T09:29:41.030944+0000","last_peered":"2024-09-12T09:29:41.030944+0000","last_clean":"2024-09-12T09:29:41.030944+0000","last_became_active":"2024-09-12T09:29:34.667951+0000","last_became_peered":"2024-09-12T09:29:34.667951+0000","last_unstale":"2024-09-12T09:29:41.030944+0000","last_undegraded":"2024-09-12T09:29:41.030944+0000","last_fullsized":"2024-09-12T09:29:41.030944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:46.198571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"33'12","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:46.011440+0000","last_change":"2024-09-12T09:29:36.686819+0000","last_active":"2024-09-12T09:29:46.011440+0000","last_peered":"2024-09-12T09:29:46.011440+0000","last_clean":"2024-09-12T09:29:46.011440+0000","last_became_active":"2024-09-12T09:29:34.675045+0000","last_became_peered":"2024-09-12T09:29:34.675045+0000","last_unstale":"2024-09-12T09:29:46.011440+0000","last_undegraded":"2024-09-12T09:29:46.011440+0000","last_fullsized":"2024-09-12T09:29:46.011440+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:11.513987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":3288812,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":80,"num_write":11,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037142+0000","last_change":"2024-09-12T09:29:36.762197+0000","last_active":"2024-09-12T09:29:42.037142+0000","last_peered":"2024-09-12T09:29:42.037142+0000","last_clean":"2024-09-12T09:29:42.037142+0000","last_became_active":"2024-09-12T09:29:34.683802+0000","last_became_peered":"2024-09-12T09:29:34.683802+0000","last_unstale":"2024-09-12T09:29:42.037142+0000","last_undegraded":"2024-09-12T09:29:42.037142+0000","last_fullsized":"2024-09-12T09:29:42.037142+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:55.340966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"28'8","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051065+0000","last_change":"2024-09-12T09:29:36.674037+0000","last_active":"2024-09-12T09:29:42.051065+0000","last_peered":"2024-09-12T09:29:42.051065+0000","last_clean":"2024-09-12T09:29:42.051065+0000","last_became_active":"2024-09-12T09:29:34.670141+0000","last_became_peered":"2024-09-12T09:29:34.670141+0000","last_unstale":"2024-09-12T09:29:42.051065+0000","last_undegraded":"2024-09-12T09:29:42.051065+0000","last_fullsized":"2024-09-12T09:29:42.051065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:40:07.866266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":2231814,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.147589+0000","last_change":"2024-09-12T09:29:26.243534+0000","last_active":"2024-09-12T09:29:45.147589+0000","last_peered":"2024-09-12T09:29:45.147589+0000","last_clean":"2024-09-12T09:29:45.147589+0000","last_became_active":"2024-09-12T09:29:23.948174+0000","last_became_peered":"2024-09-12T09:29:23.948174+0000","last_unstale":"2024-09-12T09:29:45.147589+0000","last_undegraded":"2024-09-12T09:29:45.147589+0000","last_fullsized":"2024-09-12T09:29:45.147589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:53.612149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030748+0000","last_change":"2024-09-12T09:29:36.693597+0000","last_active":"2024-09-12T09:29:41.030748+0000","last_peered":"2024-09-12T09:29:41.030748+0000","last_clean":"2024-09-12T09:29:41.030748+0000","last_became_active":"2024-09-12T09:29:34.665541+0000","last_became_peered":"2024-09-12T09:29:34.665541+0000","last_unstale":"2024-09-12T09:29:41.030748+0000","last_undegraded":"2024-09-12T09:29:41.030748+0000","last_fullsized":"2024-09-12T09:29:41.030748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:05.150134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020258+0000","last_change":"2024-09-12T09:29:25.967463+0000","last_active":"2024-09-12T09:29:41.020258+0000","last_peered":"2024-09-12T09:29:41.020258+0000","last_clean":"2024-09-12T09:29:41.020258+0000","last_became_active":"2024-09-12T09:29:23.944067+0000","last_became_peered":"2024-09-12T09:29:23.944067+0000","last_unstale":"2024-09-12T09:29:41.020258+0000","last_undegraded":"2024-09-12T09:29:41.020258+0000","last_fullsized":"2024-09-12T09:29:41.020258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:14.077402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"33'6","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.505852+0000","last_change":"2024-09-12T09:29:36.673901+0000","last_active":"2024-09-12T09:29:45.505852+0000","last_peered":"2024-09-12T09:29:45.505852+0000","last_clean":"2024-09-12T09:29:45.505852+0000","last_became_active":"2024-09-12T09:29:34.670228+0000","last_became_peered":"2024-09-12T09:29:34.670228+0000","last_unstale":"2024-09-12T09:29:45.505852+0000","last_undegraded":"2024-09-12T09:29:45.505852+0000","last_fullsized":"2024-09-12T09:29:45.505852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:42.075722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":278448,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037630+0000","last_change":"2024-09-12T09:29:26.045505+0000","last_active":"2024-09-12T09:29:42.037630+0000","last_peered":"2024-09-12T09:29:42.037630+0000","last_clean":"2024-09-12T09:29:42.037630+0000","last_became_active":"2024-09-12T09:29:24.058891+0000","last_became_peered":"2024-09-12T09:29:24.058891+0000","last_unstale":"2024-09-12T09:29:42.037630+0000","last_undegraded":"2024-09-12T09:29:42.037630+0000","last_fullsized":"2024-09-12T09:29:42.037630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:27.461463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.020839+0000","last_change":"2024-09-12T09:29:36.680262+0000","last_active":"2024-09-12T09:29:41.020839+0000","last_peered":"2024-09-12T09:29:41.020839+0000","last_clean":"2024-09-12T09:29:41.020839+0000","last_became_active":"2024-09-12T09:29:34.667442+0000","last_became_peered":"2024-09-12T09:29:34.667442+0000","last_unstale":"2024-09-12T09:29:41.020839+0000","last_undegraded":"2024-09-12T09:29:41.020839+0000","last_fullsized":"2024-09-12T09:29:41.020839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:52.348124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.021072+0000","last_change":"2024-09-12T09:29:25.967464+0000","last_active":"2024-09-12T09:29:41.021072+0000","last_peered":"2024-09-12T09:29:41.021072+0000","last_clean":"2024-09-12T09:29:41.021072+0000","last_became_active":"2024-09-12T09:29:24.058031+0000","last_became_peered":"2024-09-12T09:29:24.058031+0000","last_unstale":"2024-09-12T09:29:41.021072+0000","last_undegraded":"2024-09-12T09:29:41.021072+0000","last_fullsized":"2024-09-12T09:29:41.021072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:12.880764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"33'19","reported_seq":172,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.326721+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:29:45.326721+0000","last_peered":"2024-09-12T09:29:45.326721+0000","last_clean":"2024-09-12T09:29:45.326721+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:29:45.326721+0000","last_undegraded":"2024-09-12T09:29:45.326721+0000","last_fullsized":"2024-09-12T09:29:45.326721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:23.342932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":2915227,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":354,"num_read_kb":365,"num_write":17,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'7","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.051616+0000","last_change":"2024-09-12T09:29:36.673934+0000","last_active":"2024-09-12T09:29:42.051616+0000","last_peered":"2024-09-12T09:29:42.051616+0000","last_clean":"2024-09-12T09:29:42.051616+0000","last_became_active":"2024-09-12T09:29:34.669115+0000","last_became_peered":"2024-09-12T09:29:34.669115+0000","last_unstale":"2024-09-12T09:29:42.051616+0000","last_undegraded":"2024-09-12T09:29:42.051616+0000","last_fullsized":"2024-09-12T09:29:42.051616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:27.794886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":8,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.066478+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:29:45.066478+0000","last_peered":"2024-09-12T09:29:45.066478+0000","last_clean":"2024-09-12T09:29:45.066478+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:29:45.066478+0000","last_undegraded":"2024-09-12T09:29:45.066478+0000","last_fullsized":"2024-09-12T09:29:45.066478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"28'15","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.021094+0000","last_change":"2024-09-12T09:29:36.679214+0000","last_active":"2024-09-12T09:29:41.021094+0000","last_peered":"2024-09-12T09:29:41.021094+0000","last_clean":"2024-09-12T09:29:41.021094+0000","last_became_active":"2024-09-12T09:29:34.667231+0000","last_became_peered":"2024-09-12T09:29:34.667231+0000","last_unstale":"2024-09-12T09:29:41.021094+0000","last_undegraded":"2024-09-12T09:29:41.021094+0000","last_fullsized":"2024-09-12T09:29:41.021094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:23.576171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.057435+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:29:45.057435+0000","last_peered":"2024-09-12T09:29:45.057435+0000","last_clean":"2024-09-12T09:29:45.057435+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:29:45.057435+0000","last_undegraded":"2024-09-12T09:29:45.057435+0000","last_fullsized":"2024-09-12T09:29:45.057435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030418+0000","last_change":"2024-09-12T09:29:36.691601+0000","last_active":"2024-09-12T09:29:41.030418+0000","last_peered":"2024-09-12T09:29:41.030418+0000","last_clean":"2024-09-12T09:29:41.030418+0000","last_became_active":"2024-09-12T09:29:34.672667+0000","last_became_peered":"2024-09-12T09:29:34.672667+0000","last_unstale":"2024-09-12T09:29:41.030418+0000","last_undegraded":"2024-09-12T09:29:41.030418+0000","last_fullsized":"2024-09-12T09:29:41.030418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:53:57.721259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036530+0000","last_change":"2024-09-12T09:29:26.045329+0000","last_active":"2024-09-12T09:29:42.036530+0000","last_peered":"2024-09-12T09:29:42.036530+0000","last_clean":"2024-09-12T09:29:42.036530+0000","last_became_active":"2024-09-12T09:29:23.946961+0000","last_became_peered":"2024-09-12T09:29:23.946961+0000","last_unstale":"2024-09-12T09:29:42.036530+0000","last_undegraded":"2024-09-12T09:29:42.036530+0000","last_fullsized":"2024-09-12T09:29:42.036530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:06.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084562700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"33'2","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.258337+0000","last_change":"2024-09-12T09:29:36.673018+0000","last_active":"2024-09-12T09:29:45.258337+0000","last_peered":"2024-09-12T09:29:45.258337+0000","last_clean":"2024-09-12T09:29:45.258337+0000","last_became_active":"2024-09-12T09:29:34.656612+0000","last_became_peered":"2024-09-12T09:29:34.656612+0000","last_unstale":"2024-09-12T09:29:45.258337+0000","last_undegraded":"2024-09-12T09:29:45.258337+0000","last_fullsized":"2024-09-12T09:29:45.258337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:32:26.402801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":262160,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":3,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.066652+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:29:45.066652+0000","last_peered":"2024-09-12T09:29:45.066652+0000","last_clean":"2024-09-12T09:29:45.066652+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:29:45.066652+0000","last_undegraded":"2024-09-12T09:29:45.066652+0000","last_fullsized":"2024-09-12T09:29:45.066652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":129,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036566+0000","last_change":"2024-09-12T09:29:17.102446+0000","last_active":"2024-09-12T09:29:42.036566+0000","last_peered":"2024-09-12T09:29:42.036566+0000","last_clean":"2024-09-12T09:29:42.036566+0000","last_became_active":"2024-09-12T09:29:17.101639+0000","last_became_peered":"2024-09-12T09:29:17.101639+0000","last_unstale":"2024-09-12T09:29:42.036566+0000","last_undegraded":"2024-09-12T09:29:42.036566+0000","last_fullsized":"2024-09-12T09:29:42.036566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:11.894289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"33'121","reported_seq":814,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.974624+0000","last_change":"2024-09-12T09:29:36.673435+0000","last_active":"2024-09-12T09:29:45.974624+0000","last_peered":"2024-09-12T09:29:45.974624+0000","last_clean":"2024-09-12T09:29:45.974624+0000","last_became_active":"2024-09-12T09:29:34.665426+0000","last_became_peered":"2024-09-12T09:29:34.665426+0000","last_unstale":"2024-09-12T09:29:45.974624+0000","last_undegraded":"2024-09-12T09:29:45.974624+0000","last_fullsized":"2024-09-12T09:29:45.974624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.584108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1938,"num_read_kb":1705,"num_write":118,"num_write_kb":543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.050755+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:29:42.050755+0000","last_peered":"2024-09-12T09:29:42.050755+0000","last_clean":"2024-09-12T09:29:42.050755+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:29:42.050755+0000","last_undegraded":"2024-09-12T09:29:42.050755+0000","last_fullsized":"2024-09-12T09:29:42.050755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:31.590981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":4115751,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"33'12","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.994576+0000","last_change":"2024-09-12T09:29:36.727472+0000","last_active":"2024-09-12T09:29:45.994576+0000","last_peered":"2024-09-12T09:29:45.994576+0000","last_clean":"2024-09-12T09:29:45.994576+0000","last_became_active":"2024-09-12T09:29:34.662340+0000","last_became_peered":"2024-09-12T09:29:34.662340+0000","last_unstale":"2024-09-12T09:29:45.994576+0000","last_undegraded":"2024-09-12T09:29:45.994576+0000","last_fullsized":"2024-09-12T09:29:45.994576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:53.890918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":417792,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":12,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"33'13","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.643765+0000","last_change":"2024-09-12T09:29:36.674453+0000","last_active":"2024-09-12T09:29:45.643765+0000","last_peered":"2024-09-12T09:29:45.643765+0000","last_clean":"2024-09-12T09:29:45.643765+0000","last_became_active":"2024-09-12T09:29:34.671250+0000","last_became_peered":"2024-09-12T09:29:34.671250+0000","last_unstale":"2024-09-12T09:29:45.643765+0000","last_undegraded":"2024-09-12T09:29:45.643765+0000","last_fullsized":"2024-09-12T09:29:45.643765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:28:04.510986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":19520705,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":63,"num_read_kb":144,"num_write":11,"num_write_kb":653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"33'3","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.485461+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:29:45.485461+0000","last_peered":"2024-09-12T09:29:45.485461+0000","last_clean":"2024-09-12T09:29:45.485461+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:29:45.485461+0000","last_undegraded":"2024-09-12T09:29:45.485461+0000","last_fullsized":"2024-09-12T09:29:45.485461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:24.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037055+0000","last_change":"2024-09-12T09:29:36.763498+0000","last_active":"2024-09-12T09:29:42.037055+0000","last_peered":"2024-09-12T09:29:42.037055+0000","last_clean":"2024-09-12T09:29:42.037055+0000","last_became_active":"2024-09-12T09:29:34.678236+0000","last_became_peered":"2024-09-12T09:29:34.678236+0000","last_unstale":"2024-09-12T09:29:42.037055+0000","last_undegraded":"2024-09-12T09:29:42.037055+0000","last_fullsized":"2024-09-12T09:29:42.037055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:34:56.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"28'6","reported_seq":36,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037081+0000","last_change":"2024-09-12T09:29:36.762465+0000","last_active":"2024-09-12T09:29:42.037081+0000","last_peered":"2024-09-12T09:29:42.037081+0000","last_clean":"2024-09-12T09:29:42.037081+0000","last_became_active":"2024-09-12T09:29:34.681589+0000","last_became_peered":"2024-09-12T09:29:34.681589+0000","last_unstale":"2024-09-12T09:29:42.037081+0000","last_undegraded":"2024-09-12T09:29:42.037081+0000","last_fullsized":"2024-09-12T09:29:42.037081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:23.162912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"28'9","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037301+0000","last_change":"2024-09-12T09:29:36.763375+0000","last_active":"2024-09-12T09:29:42.037301+0000","last_peered":"2024-09-12T09:29:42.037301+0000","last_clean":"2024-09-12T09:29:42.037301+0000","last_became_active":"2024-09-12T09:29:34.672883+0000","last_became_peered":"2024-09-12T09:29:34.672883+0000","last_unstale":"2024-09-12T09:29:42.037301+0000","last_undegraded":"2024-09-12T09:29:42.037301+0000","last_fullsized":"2024-09-12T09:29:42.037301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:50:45.246121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"33'32","reported_seq":732,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-12T09:29:45.923442+0000","last_change":"2024-09-12T09:29:36.727208+0000","last_active":"2024-09-12T09:29:45.923442+0000","last_peered":"2024-09-12T09:29:45.923442+0000","last_clean":"2024-09-12T09:29:45.923442+0000","last_became_active":"2024-09-12T09:29:34.674419+0000","last_became_peered":"2024-09-12T09:29:34.674419+0000","last_unstale":"2024-09-12T09:29:45.923442+0000","last_undegraded":"2024-09-12T09:29:45.923442+0000","last_fullsized":"2024-09-12T09:29:45.923442+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:48:21.450188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":18199795,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":686,"num_read_kb":5381,"num_write":32,"num_write_kb":10788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'1","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.037646+0000","last_change":"2024-09-12T09:29:36.761814+0000","last_active":"2024-09-12T09:29:42.037646+0000","last_peered":"2024-09-12T09:29:42.037646+0000","last_clean":"2024-09-12T09:29:42.037646+0000","last_became_active":"2024-09-12T09:29:34.675375+0000","last_became_peered":"2024-09-12T09:29:34.675375+0000","last_unstale":"2024-09-12T09:29:42.037646+0000","last_undegraded":"2024-09-12T09:29:42.037646+0000","last_fullsized":"2024-09-12T09:29:42.037646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:13.413928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-12T09:29:41.030422+0000","last_change":"2024-09-12T09:29:36.691565+0000","last_active":"2024-09-12T09:29:41.030422+0000","last_peered":"2024-09-12T09:29:41.030422+0000","last_clean":"2024-09-12T09:29:41.030422+0000","last_became_active":"2024-09-12T09:29:34.670997+0000","last_became_peered":"2024-09-12T09:29:34.670997+0000","last_unstale":"2024-09-12T09:29:41.030422+0000","last_undegraded":"2024-09-12T09:29:41.030422+0000","last_fullsized":"2024-09-12T09:29:41.030422+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:21.434542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"28'9","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-12T09:29:42.036838+0000","last_change":"2024-09-12T09:29:36.762929+0000","last_active":"2024-09-12T09:29:42.036838+0000","last_peered":"2024-09-12T09:29:42.036838+0000","last_clean":"2024-09-12T09:29:42.036838+0000","last_became_active":"2024-09-12T09:29:34.673272+0000","last_became_peered":"2024-09-12T09:29:34.673272+0000","last_unstale":"2024-09-12T09:29:42.036838+0000","last_undegraded":"2024-09-12T09:29:42.036838+0000","last_fullsized":"2024-09-12T09:29:42.036838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:20.910614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":76858160,"num_objects":53,"num_object_clones":4,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":2,"num_read":5023,"num_read_kb":13914,"num_write":743,"num_write_kb":25394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":9658147,"data_compressed":486276,"data_compressed_allocated":1089536,"data_compressed_original":9416704,"omap_allocated":0,"internal_metadata":0},"log_size":608,"ondisk_log_size":608,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":11678,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":820048,"data_compressed":54440,"data_compressed_allocated":94208,"data_compressed_original":667648,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":450560,"data_stored":314538,"data_compressed":81867,"data_compressed_allocated":94208,"data_compressed_original":167936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":1375542,"data_compressed":77271,"data_compressed_allocated":151552,"data_compressed_original":1212416,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31752,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":573440,"data_stored":2359293,"data_compressed":1268,"data_compressed_allocated":143360,"data_compressed_original":2187264,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":684032,"data_stored":537966,"data_compressed":184161,"data_compressed_allocated":208896,"data_compressed_original":335872,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":442368,"data_stored":498439,"data_compressed":36227,"data_compressed_allocated":57344,"data_compressed_original":339968,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":327680,"data_stored":164206,"data_compressed":120,"data_compressed_allocated":8192,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35820,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336020,"statfs":{"total":96636764160,"available":96600084480,"internally_reserved":0,"allocated":487424,"data_stored":891292,"data_compressed":77483,"data_compressed_allocated":122880,"data_compressed_original":745472,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":25442,"data_compressed":12808,"data_compressed_allocated":16384,"data_compressed_original":20480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":51079,"data_compressed":120,"data_compressed_allocated":8192,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":1666298,"data_compressed":37086,"data_compressed_allocated":139264,"data_compressed_original":1650688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":2797464,"data_compressed":216178,"data_compressed_allocated":380928,"data_compressed_original":2707456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":195321,"data_compressed":81867,"data_compressed_allocated":94208,"data_compressed_original":167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":3289215,"data_compressed":12161,"data_compressed_allocated":221184,"data_compressed_original":3252224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":1522865,"data_compressed":77455,"data_compressed_allocated":172032,"data_compressed_original":1482752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":110463,"data_compressed":48601,"data_compressed_allocated":57344,"data_compressed_original":94208,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:29:46.797 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T09:29:46.797 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:29:46.797 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:29:46.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-12T09:29:46.799 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:46.799 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:46.801 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:46.801 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:46.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:46.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:46.829 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:46.841 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:46.838+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:46.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:46.870 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:46.942 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:46.938+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:46.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:47.055 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:47.054+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:47.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:29:47.114 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:29:47.110+0000 7f3454e55640 -1 mon.a@0(leader).osd e35 definitely_dead 0 2024-09-12T09:29:47.127 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.127 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:47.146 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:47.142+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:47.249 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:47.246+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:47.312 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.313 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:47.350 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:47.346+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:47.353 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-12T09:29:47.356 INFO:teuthology.orchestra.run.smithi007.stdout:670 write 0xc2cb131 thru 0xc2cc043 (0xf13 bytes) 2024-09-12T09:29:47.370 INFO:teuthology.orchestra.run.smithi007.stdout:671 write 0x40dd2d4 thru 0x40dddfc (0xb29 bytes) 2024-09-12T09:29:47.398 INFO:teuthology.orchestra.run.smithi007.stdout:672 trunc from 0xe09aeaf to 0x1225731 2024-09-12T09:29:47.453 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:47.450+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:47.453 INFO:teuthology.orchestra.run.smithi007.stdout:673 write 0x3792bab thru 0x379f2b0 (0xc706 bytes) 2024-09-12T09:29:47.472 INFO:teuthology.orchestra.run.smithi007.stdout:674 writesame 0x53c8795 thru 0x53d058e data_size 0x81(0x7dfa bytes) 2024-09-12T09:29:47.476 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.476 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:47.492 INFO:teuthology.orchestra.run.smithi007.stdout:675 writesame 0xb988257 thru 0xb98a992 data_size 0x51(0x273c bytes) 2024-09-12T09:29:47.493 INFO:teuthology.orchestra.run.smithi007.stdout:676 read 0x9da034b thru 0x9daa6c6 (0xa37c bytes) 2024-09-12T09:29:47.508 INFO:teuthology.orchestra.run.smithi007.stdout:678 write 0xd1bff89 thru 0xd1ce544 (0xe5bc bytes) 2024-09-12T09:29:47.515 INFO:teuthology.orchestra.run.smithi007.stdout:680 punch from 0x6baf431 to 0x6bb1c21, (0x27f0 bytes) 2024-09-12T09:29:47.515 INFO:teuthology.orchestra.run.smithi007.stdout:682 read 0x7d352f1 thru 0x7d3f875 (0xa585 bytes) 2024-09-12T09:29:47.515 INFO:teuthology.orchestra.run.smithi007.stdout:684 read 0x664d5e2 thru 0x6659ca7 (0xc6c6 bytes) 2024-09-12T09:29:47.516 INFO:teuthology.orchestra.run.smithi007.stdout:685 read 0x8c7a0f3 thru 0x8c8829c (0xe1aa bytes) 2024-09-12T09:29:47.516 INFO:teuthology.orchestra.run.smithi007.stdout:686 read 0x430d6c thru 0x43d902 (0xcb97 bytes) 2024-09-12T09:29:47.516 INFO:teuthology.orchestra.run.smithi007.stdout:687 read 0x3abe44d thru 0x3ac816a (0x9d1e bytes) 2024-09-12T09:29:47.531 INFO:teuthology.orchestra.run.smithi007.stdout:688 trunc from 0xd1ce545 to 0x5e320dd 2024-09-12T09:29:47.543 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:47.553 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:47.550+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:47.558 INFO:teuthology.orchestra.run.smithi007.stdout:690 write 0x8f4027f thru 0x8f42bd7 (0x2959 bytes) 2024-09-12T09:29:47.564 INFO:teuthology.orchestra.run.smithi007.stdout:692 read 0xdd83cf thru 0xddd992 (0x55c4 bytes) 2024-09-12T09:29:47.565 INFO:teuthology.orchestra.run.smithi007.stdout:693 write 0x48bb2f4 thru 0x48c6f9c (0xbca9 bytes) 2024-09-12T09:29:47.576 INFO:teuthology.orchestra.run.smithi007.stdout:694 trunc from 0x8f42bd8 to 0x4c96ddb 2024-09-12T09:29:47.604 INFO:teuthology.orchestra.run.smithi007.stdout:696 trunc from 0x4c96ddb to 0x80f7a38 2024-09-12T09:29:47.616 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:47.635 INFO:teuthology.orchestra.run.smithi007.stdout:700 writesame 0xacd0ef3 thru 0xacdc05d data_size 0x1(0xb16b bytes) 2024-09-12T09:29:47.635 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.635 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:47.651 INFO:teuthology.orchestra.run.smithi007.stdout:701 write 0xb780f26 thru 0xb788f3c (0x8017 bytes) 2024-09-12T09:29:47.656 INFO:teuthology.orchestra.run.smithi007.stdout:702 read 0x77590f3 thru 0x775bb08 (0x2a16 bytes) 2024-09-12T09:29:47.657 INFO:teuthology.orchestra.run.smithi007.stdout:703 write 0x6542cd4 thru 0x654d380 (0xa6ad bytes) 2024-09-12T09:29:47.661 INFO:teuthology.orchestra.run.smithi007.stdout:704 punch from 0x9cc4437 to 0x9cc87b3, (0x437c bytes) 2024-09-12T09:29:47.662 INFO:teuthology.orchestra.run.smithi007.stdout:706 read 0x808d379 thru 0x809ccb2 (0xf93a bytes) 2024-09-12T09:29:47.677 INFO:teuthology.orchestra.run.smithi007.stdout:709 writesame 0xdeef8ff thru 0xdefaf1a data_size 0xb9(0xb61c bytes) 2024-09-12T09:29:47.679 INFO:teuthology.orchestra.run.smithi007.stdout:710 write 0x434477b thru 0x4348460 (0x3ce6 bytes) 2024-09-12T09:29:47.681 INFO:teuthology.orchestra.run.smithi007.stdout:712 write 0x70a49ad thru 0x70b1edc (0xd530 bytes) 2024-09-12T09:29:47.686 INFO:teuthology.orchestra.run.smithi007.stdout:713 clone 5 order 21 su 16384 sc 4 2024-09-12T09:29:47.694 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:47.694+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:47.758 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:47.754+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:47.769 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.769 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:47.875 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:47.874+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:47.948 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:47.948 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:47.960 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:47.958+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:47.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:48.061 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:48.058+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:29:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.0. 2024-09-12T09:29:48.138 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.138 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:48.161 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:48.158+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:48.262 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:48.258+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:48.272 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.272 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:48.363 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:48.358+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:48.460 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.460 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.463 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:48.462+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:48.477 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:48.565 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:48.561+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:48.602 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.602 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:48.665 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:48.662+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:48.739 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.739 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.753 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:48.766 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:48.761+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:48.869 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:48.865+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:48.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:48.917 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:48.917 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:48.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:48.968 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:48.965+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:49.049 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:49.050 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:49.052 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:49.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:49.069 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:49.065+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:49.072 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:49.073 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:49.097 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:49.097 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:49.119 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:49.210 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone4 2024-09-12T09:29:49.211 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:49.211 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:49.211 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:49.212 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:49.212 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:49.212 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:49.213 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:49.213 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:49.205+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:49.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:49.308 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-12T09:29:49.311 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:49.310+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:49.342 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2474 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14049280~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397377+0000", 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9457833330000001, 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047890099999999998, 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2474 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397377+0000", 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397377+0000", 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397378+0000", 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.514e-06 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397385+0000", 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397403+0000", 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8085999999999999e-05 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397417+0000", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4054e-05 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.444992+0000", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047575144999999999 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445116+0000", 2024-09-12T09:29:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124246 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445267+0000", 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015063199999999999 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2475 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14082048~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397439+0000", 2024-09-12T09:29:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9457205960000001, 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154638, 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2475 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397439+0000", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397439+0000", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397441+0000", 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3829999999999999e-06 2024-09-12T09:29:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397446+0000", 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3970000000000002e-06 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397464+0000", 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7672999999999998e-05 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397477+0000", 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3339999999999999e-05 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445335+0000", 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047857980000000001 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445454+0000", 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118572 2024-09-12T09:29:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445594+0000", 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014029299999999999 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2476 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14114816~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397496+0000", 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9456644700000001, 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048795357999999997, 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2476 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397496+0000", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397496+0000", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397497+0000", 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.133e-06 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397502+0000", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4639999999999997e-06 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397520+0000", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7649e-05 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397533+0000", 2024-09-12T09:29:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3550000000000001e-05 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445720+0000", 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048186336000000003 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.445972+0000", 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025206099999999998 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446291+0000", 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000319165 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2477 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14147584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397552+0000", 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9456081809999999, 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049098240000000001, 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2477 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397552+0000", 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397552+0000", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397553+0000", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1939999999999999e-06 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397558+0000", 2024-09-12T09:29:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2900000000000002e-06 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397575+0000", 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6685999999999999e-05 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397589+0000", 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4134000000000001e-05 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446339+0000", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048749608 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446467+0000", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012831600000000001 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446650+0000", 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000183012 2024-09-12T09:29:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9455530009999999, 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:29:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:29:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.945493591, 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:29:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9454370050000001, 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:29:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:29:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9453686919999997, 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:29:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9453105060000002, 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:29:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:29:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9452499730000001, 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:29:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:29:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9451899729999997, 2024-09-12T09:29:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:29:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9451336239999999, 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:29:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9450743140000002, 2024-09-12T09:29:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:29:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9450186810000001, 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:29:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:29:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:29:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:29:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9449622980000001, 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:29:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:29:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9449057280000002, 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:29:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9448493120000001, 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:29:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:29:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.944793819, 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:29:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:29:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:29:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.944705806, 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:29:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.9446413630000001, 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:29:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:29:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:29:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:29:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:29:49.442 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:49.443 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:49.442+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:49.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:49.545 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:49.546+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:49.565 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:49.565 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:49.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:49.646 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:49.642+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:49.703 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:49.704 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:49.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:49.747 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:49.746+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:49.848 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:49.846+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2919 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 942080~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.2712186680000004, 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041365445000000001, 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2919 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575880+0000", 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7500000000000001e-07 2024-09-12T09:29:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575881+0000", 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575884+0000", 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4060000000000001e-06 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575887+0000", 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7930000000000002e-06 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616841+0000", 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040954162000000002 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616963+0000", 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012160800000000001 2024-09-12T09:29:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617245+0000", 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028208299999999998 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2920 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 974848~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.271207135, 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041906318999999997, 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2920 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:29:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575892+0000", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0950000000000001e-06 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575895+0000", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575902+0000", 2024-09-12T09:29:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4489999999999996e-06 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617280+0000", 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041377835000000002 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617372+0000", 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2070000000000004e-05 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617797+0000", 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000425284 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2921 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1007616~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.271190432, 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042305425000000001, 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2921 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575909+0000", 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429999999999999e-06 2024-09-12T09:29:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575912+0000", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5659999999999998e-06 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575915+0000", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8880000000000001e-06 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617854+0000", 2024-09-12T09:29:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041939002000000003 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617933+0000", 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8893000000000005e-05 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618213+0000", 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000279671 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2922 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1040384~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.2711784120000003, 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042845688999999999, 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2922 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575921+0000", 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0139999999999999e-06 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575924+0000", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3330000000000001e-06 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575927+0000", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6469999999999998e-06 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618438+0000", 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042511372999999998 2024-09-12T09:29:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618552+0000", 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114024 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618765+0000", 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213069 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5493 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0983632849999996, 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041875908000000003, 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5493 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748736+0000", 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1939999999999999e-06 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748746+0000", 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5969999999999998e-06 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748752+0000", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790107+0000", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041354634000000001 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790210+0000", 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010341899999999999 2024-09-12T09:29:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790611+0000", 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000400212 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5494 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0983376680000001, 2024-09-12T09:29:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042226430000000002, 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5494 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748761+0000", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748762+0000", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.739e-06 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748767+0000", 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1609999999999996e-06 2024-09-12T09:29:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748770+0000", 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9500000000000001e-06 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790647+0000", 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041877437000000003 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790753+0000", 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105726 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790987+0000", 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023401800000000001 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5495 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0983128049999999, 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042594770999999997, 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5495 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748787+0000", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2810000000000001e-06 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748796+0000", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748801+0000", 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3850000000000001e-06 2024-09-12T09:29:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791018+0000", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042216562999999999 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791093+0000", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5080999999999995e-05 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791380+0000", 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028708999999999999 2024-09-12T09:29:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5496 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2744320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.098292152, 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042916190999999999, 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5496 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.35e-07 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748807+0000", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.237e-06 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748810+0000", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2210000000000002e-06 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748813+0000", 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5959999999999998e-06 2024-09-12T09:29:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791412+0000", 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042598534 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791488+0000", 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5994000000000005e-05 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791722+0000", 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234374 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5497 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0982809759999999, 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043287796000000003, 2024-09-12T09:29:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5497 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748818+0000", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.113e-06 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748828+0000", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9019999999999994e-06 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748835+0000", 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4400000000000002e-06 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791754+0000", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042919481000000002 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791836+0000", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1514000000000006e-05 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792105+0000", 2024-09-12T09:29:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000269125 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5498 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.098247583, 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043631883000000003, 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5498 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748851+0000", 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748852+0000", 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7680000000000001e-06 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748857+0000", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5510000000000001e-06 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748860+0000", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.157e-06 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792142+0000", 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043281687999999999 2024-09-12T09:29:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792246+0000", 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103749 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792482+0000", 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023665099999999999 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5499 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2842624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0982333339999997, 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043996252, 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5499 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748885+0000", 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0276999999999999e-05 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748894+0000", 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7809999999999994e-06 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748898+0000", 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6669999999999998e-06 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792530+0000", 2024-09-12T09:29:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043632399000000002 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792612+0000", 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2317999999999993e-05 2024-09-12T09:29:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792861+0000", 2024-09-12T09:29:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024859200000000001 2024-09-12T09:29:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5500 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981937039999998, 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044345271999999998, 2024-09-12T09:29:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5500 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748905+0000", 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748906+0000", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2300000000000001e-06 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748910+0000", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2289999999999997e-06 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748913+0000", 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7439999999999999e-06 2024-09-12T09:29:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792921+0000", 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044007743000000002 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793009+0000", 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8284999999999999e-05 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793250+0000", 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000240673 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5501 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2908160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981812360000003, 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044652050999999998, 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5501 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748918+0000", 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.128e-06 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748921+0000", 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9890000000000001e-06 2024-09-12T09:29:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748924+0000", 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793279+0000", 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044355798000000002 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793363+0000", 2024-09-12T09:29:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3100000000000001e-05 2024-09-12T09:29:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793569+0000", 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020620799999999999 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5502 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981707109999999, 2024-09-12T09:29:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045033795000000001, 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5502 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748929+0000", 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2610000000000001e-06 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748934+0000", 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2340000000000002e-06 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748938+0000", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2640000000000002e-06 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793678+0000", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044739745999999997 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793730+0000", 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909000000000001e-05 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793961+0000", 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023114199999999999 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5503 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981531039999997, 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045434384000000001, 2024-09-12T09:29:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5503 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748947+0000", 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.362e-06 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748951+0000", 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5539999999999999e-06 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748954+0000", 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794016+0000", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045062229000000002 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794103+0000", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7168000000000003e-05 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794379+0000", 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027592900000000001 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5504 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981398919999998, 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045942016000000002, 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5504 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1800000000000002e-07 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748960+0000", 2024-09-12T09:29:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.779e-06 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748966+0000", 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5650000000000001e-06 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748971+0000", 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794420+0000", 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045449472999999997 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794513+0000", 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2631000000000001e-05 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794900+0000", 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038728900000000003 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5505 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0981071839999998, 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046773908000000003, 2024-09-12T09:29:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5505 2024-09-12T09:29:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8700000000000005e-07 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748994+0000", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749020+0000", 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6032000000000001e-05 2024-09-12T09:29:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749028+0000", 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794934+0000", 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045905796999999998 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795020+0000", 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5636000000000005e-05 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795765+0000", 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074500200000000003 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5506 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0980580680000003, 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047076258000000003, 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5506 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749043+0000", 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7879999999999998e-06 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749057+0000", 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4277000000000001e-05 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749067+0000", 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999999e-06 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795798+0000", 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046730657000000002 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795882+0000", 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4432999999999996e-05 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796116+0000", 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023388599999999999 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5507 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0980209649999999, 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047404593000000002, 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5507 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749080+0000", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7159999999999999e-06 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749088+0000", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0250000000000002e-06 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749094+0000", 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-12T09:29:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796153+0000", 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047059813999999998 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796244+0000", 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0463000000000004e-05 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796482+0000", 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0002377 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5508 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0979967630000003, 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047813034999999997, 2024-09-12T09:29:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5508 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749102+0000", 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749104+0000", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749110+0000", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749116+0000", 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5889999999999999e-06 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796574+0000", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047457634999999998 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796658+0000", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4040000000000002e-05 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796914+0000", 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000256238 2024-09-12T09:29:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:49.953 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:49.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:49.955 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:49.954+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:50.056 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:50.054+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:50.086 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:50.086 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:50.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:50.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:29:50.196 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:50.193+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:50.233 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:50.234 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:50.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:50.258 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:50.254+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:50.359 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:50.357+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:50.423 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.371182385999999, 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:29:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6506306019999997, 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:29:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6491266099999997, 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:29:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6491038610000004, 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:29:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6490804209999999, 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:29:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:29:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6490569730000004, 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:29:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:29:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6490135370000001, 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:29:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:29:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:29:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:29:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6489767400000002, 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:29:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5809885000000001, 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:29:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5809329400000003, 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:29:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5809036909999996, 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:29:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:29:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5808799870000003, 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:29:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5808602589999996, 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:29:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:29:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5808367370000003, 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:29:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5808137630000001, 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:29:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:29:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5807695739999996, 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:29:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.580735207, 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:29:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5273118380000001, 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:29:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8464 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5201920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5267575449999997, 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016641282, 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8464 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894359+0000", 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894360+0000", 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894364+0000", 2024-09-12T09:29:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0879999999999999e-06 2024-09-12T09:29:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894366+0000", 2024-09-12T09:29:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.469e-06 2024-09-12T09:29:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.910874+0000", 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016508228999999999 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.910955+0000", 2024-09-12T09:29:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1094000000000007e-05 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911000+0000", 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4660000000000003e-05 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5267470020000005, 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:29:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:29:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:50.527 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:50.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:714 punch from 0xc4c1522 to 0xc4c281c, (0x12fa bytes) 2024-09-12T09:29:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:716 trunc from 0xdefaf1b to 0x8057aa3 2024-09-12T09:29:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:718 read 0x62e185 thru 0x62fee3 (0x1d5f bytes) 2024-09-12T09:29:50.530 INFO:teuthology.orchestra.run.smithi007.stdout:719 writesame 0x56ce65e thru 0x56d9fff data_size 0x1(0xb9a2 bytes) 2024-09-12T09:29:50.530 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:50.525+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:50.565 INFO:teuthology.orchestra.run.smithi007.stdout:720 write 0xb61a2eb thru 0xb6208bc (0x65d2 bytes) 2024-09-12T09:29:50.595 INFO:teuthology.orchestra.run.smithi007.stdout:721 writesame 0xd52ae50 thru 0xd5399da data_size 0x1(0xeb8b bytes) 2024-09-12T09:29:50.599 INFO:teuthology.orchestra.run.smithi007.stdout:722 trunc from 0xd5399db to 0xbb25d0d 2024-09-12T09:29:50.612 INFO:teuthology.orchestra.run.smithi007.stdout:723 writesame 0x89adad3 thru 0x89bb566 data_size 0x1(0xda94 bytes) 2024-09-12T09:29:50.630 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:50.625+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:50.645 INFO:teuthology.orchestra.run.smithi007.stdout:724 read 0x6b68874 thru 0x6b74700 (0xbe8d bytes) 2024-09-12T09:29:50.648 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:50.648 INFO:teuthology.orchestra.run.smithi007.stdout:726 write 0x9cdf901 thru 0x9ce761d (0x7d1d bytes) 2024-09-12T09:29:50.660 INFO:teuthology.orchestra.run.smithi007.stdout:727 write 0x9c703d7 thru 0x9c7e8b0 (0xe4da bytes) 2024-09-12T09:29:50.672 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:50.685 INFO:teuthology.orchestra.run.smithi007.stdout:730 trunc from 0xbb25d0d to 0x3e5d750 2024-09-12T09:29:50.701 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:50.702 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:50.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:50.730 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:50.726+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:50.812 INFO:teuthology.orchestra.run.smithi007.stdout:731 writesame 0x83d6f48 thru 0x83e15d8 data_size 0x1(0xa691 bytes) 2024-09-12T09:29:50.812 INFO:teuthology.orchestra.run.smithi007.stdout:732 write 0x3e6fe20 thru 0x3e73230 (0x3411 bytes) 2024-09-12T09:29:50.816 INFO:teuthology.orchestra.run.smithi007.stdout:733 read 0x3ca2466 thru 0x3ca3c34 (0x17cf bytes) 2024-09-12T09:29:50.831 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:50.830+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:50.839 INFO:teuthology.orchestra.run.smithi007.stdout:734 writesame 0xe6673db thru 0xe670eea data_size 0x18(0x9b10 bytes) 2024-09-12T09:29:50.839 INFO:teuthology.orchestra.run.smithi007.stdout:735 write 0x3c54278 thru 0x3c5560b (0x1394 bytes) 2024-09-12T09:29:50.842 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:50.842 INFO:teuthology.orchestra.run.smithi007.stdout:738 writesame 0xcfa3f8 thru 0xcfa797 data_size 0x8(0x3a0 bytes) 2024-09-12T09:29:50.885 INFO:teuthology.orchestra.run.smithi007.stdout:739 write 0x34c8455 thru 0x34cf86d (0x7419 bytes) 2024-09-12T09:29:50.892 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:50.893 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:50.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:50.932 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:50.930+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:50.934 INFO:teuthology.orchestra.run.smithi007.stdout:741 write 0x3fe3de5 thru 0x3fef1fc (0xb418 bytes) 2024-09-12T09:29:50.940 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:50.940 INFO:teuthology.orchestra.run.smithi007.stdout:746 read 0x42e6cb7 thru 0x42ed5e1 (0x692b bytes) 2024-09-12T09:29:50.954 INFO:teuthology.orchestra.run.smithi007.stdout:747 trunc from 0xe670eeb to 0x4fd756a 2024-09-12T09:29:50.984 INFO:teuthology.orchestra.run.smithi007.stdout:748 write 0x73d76f5 thru 0x73e2da7 (0xb6b3 bytes) 2024-09-12T09:29:51.008 INFO:teuthology.orchestra.run.smithi007.stdout:749 trunc from 0x73e2da8 to 0xc42fbc6 2024-09-12T09:29:51.021 INFO:teuthology.orchestra.run.smithi007.stdout:751 write 0x3e412d8 thru 0x3e4c933 (0xb65c bytes) 2024-09-12T09:29:51.028 INFO:teuthology.orchestra.run.smithi007.stdout:752 write 0x52d6574 thru 0x52e3621 (0xd0ae bytes) 2024-09-12T09:29:51.033 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:51.029+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:51.041 INFO:teuthology.orchestra.run.smithi007.stdout:753 writesame 0x7593e57 thru 0x75a3db2 data_size 0x76(0xff5c bytes) 2024-09-12T09:29:51.046 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1015 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'4, mlcod=27'2)", 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.622792486, 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001427214, 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422637+0000", 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422646+0000", 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7080000000000003e-06 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422678+0000", 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1829000000000003e-05 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422692+0000", 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4116e-05 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422811+0000", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011919899999999999 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422852+0000", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1016000000000002e-05 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.423982+0000", 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011297950000000001 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.424063+0000", 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1023e-05 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.57994907, 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:29:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:29:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:29:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1039 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'13, mlcod=28'11)", 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.559710718, 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013168500000000001, 2024-09-12T09:29:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485720+0000", 2024-09-12T09:29:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1670000000000002e-06 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485733+0000", 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3183000000000001e-05 2024-09-12T09:29:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485765+0000", 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1866000000000003e-05 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485785+0000", 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9830000000000002e-05 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485917+0000", 2024-09-12T09:29:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013222599999999999 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485968+0000", 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1106000000000002e-05 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.486960+0000", 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099206899999999994 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487034+0000", 2024-09-12T09:29:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4402999999999999e-05 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.515285445, 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:29:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.510716563000001, 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:29:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.333055151, 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:29:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.310096786000001, 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:29:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:29:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8797483670000004, 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:3734 3.4 e33/23 3:21777318:::rbd_data.11912e4e65e9.0000000000000017:head v 33'1, mlcod=0'0)", 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.396540313, 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001459442, 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648890+0000", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.829e-06 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649057+0000", 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729099999999999 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649076+0000", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8902e-05 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649083+0000", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.277e-06 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649153+0000", 2024-09-12T09:29:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9380000000000003e-05 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649177+0000", 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.425e-05 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650298+0000", 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011215369999999999 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650347+0000", 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8976e-05 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:4220 3.4 e33/23 3:245d796b:::rbd_data.11912e4e65e9.0000000000000026:head v 33'2, mlcod=0'0)", 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3636351600000003, 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014342420000000001, 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681795+0000", 2024-09-12T09:29:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681925+0000", 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012997400000000001 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681944+0000", 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9531e-05 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681954+0000", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5149999999999995e-06 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682029+0000", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5421999999999997e-05 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682056+0000", 2024-09-12T09:29:51.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6942000000000001e-05 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683173+0000", 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001117093 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683227+0000", 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000002e-05 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5962 3.4 e33/23 3:21146361:::rbd_data.11912e4e65e9.000000000000005d:head v 33'3, mlcod=33'1)", 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2623656619999997, 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001433824, 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783064+0000", 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.798e-06 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783189+0000", 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012434600000000001 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783217+0000", 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8218e-05 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783230+0000", 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2965e-05 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783343+0000", 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113309 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783366+0000", 2024-09-12T09:29:51.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2625000000000001e-05 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784454+0000", 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001088356 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784496+0000", 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2206999999999998e-05 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5964 3.0 e33/23 3:02a4fae5:::rbd_data.11912e4e65e9.000000000000005f:head v 33'32, mlcod=31'29)", 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.254488813, 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001299197, 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790941+0000", 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5510000000000001e-06 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791013+0000", 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2242000000000004e-05 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791043+0000", 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0028000000000001e-05 2024-09-12T09:29:51.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791053+0000", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0154e-05 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791195+0000", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014127599999999999 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791216+0000", 2024-09-12T09:29:51.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-05 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792186+0000", 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096983200000000001 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792239+0000", 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2814e-05 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2509750459999998, 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:29:51.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:29:51.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2501994139999999, 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:9124 3.0 e34/23 3:01b4a15d:::rbd_data.11912e4e65e9.0000000000000122:head v 34'34, mlcod=33'32)", 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:46.145607+0000", 2024-09-12T09:29:51.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8998214220000005, 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001140561, 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145607+0000", 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145607+0000", 2024-09-12T09:29:51.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145608+0000", 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1800000000000004e-07 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145634+0000", 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6077000000000001e-05 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145649+0000", 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4796e-05 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145656+0000", 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9890000000000003e-06 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145749+0000", 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3005999999999997e-05 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.145768+0000", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.895e-05 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.146715+0000", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094764700000000003 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:46.146747+0000", 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2178000000000003e-05 2024-09-12T09:29:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91428330400000002, 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:29:51.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:29:51.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90747462700000003, 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:29:51.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.51852706100000001, 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:29:51.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:29:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.51515702799999996, 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:29:51.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10684 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'7, mlcod=34'5)", 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.690557+0000", 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.35487144599999998, 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011587839999999999, 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690557+0000", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690557+0000", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690559+0000", 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6570000000000001e-06 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690576+0000", 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6935e-05 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690612+0000", 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6043e-05 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690634+0000", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1348999999999998e-05 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690744+0000", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010984000000000001 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.690772+0000", 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8396999999999999e-05 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.691590+0000", 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081837499999999998 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.691716+0000", 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012518800000000001 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:51.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:51.146 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:51.146 INFO:teuthology.orchestra.run.smithi007.stdout:754 read 0x7e0ba96 thru 0x7e0e058 (0x25c3 bytes) 2024-09-12T09:29:51.146 INFO:teuthology.orchestra.run.smithi007.stdout:755 trunc from 0xc42fbc6 to 0x849e9e3 2024-09-12T09:29:51.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:51.148 INFO:teuthology.orchestra.run.smithi007.stdout:756 writesame 0xd4eed11 thru 0xd4f94fe data_size 0xf(0xa7ee bytes) 2024-09-12T09:29:51.148 INFO:teuthology.orchestra.run.smithi007.stdout:757 punch from 0x6546f57 to 0x655244a, (0xb4f3 bytes) 2024-09-12T09:29:51.148 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:51.148 INFO:teuthology.orchestra.run.smithi007.stdout:761 trunc from 0xd4f94ff to 0xa25e05b 2024-09-12T09:29:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:29:51.149 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:51.149 INFO:teuthology.orchestra.run.smithi007.stdout:763 writesame 0x829bbed thru 0x82a509b data_size 0x1(0x94af bytes) 2024-09-12T09:29:51.151 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:51.152 INFO:teuthology.orchestra.run.smithi007.stdout:765 clone 6 order 19 su 4096 sc 6 2024-09-12T09:29:51.152 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:51.149+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:51.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:51.263 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:51.261+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:51.278 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:51.278 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:51.278 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:51.278 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:51.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:51.353 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:51.350+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:51.364 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:51.454 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:51.454+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:51.456 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:51.456 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:51.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:51.555 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:51.554+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:51.587 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:51.624 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:51.656 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:51.653+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:51.657 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:51.668 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:51.668 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:51.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:51.692 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:51.712 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:51.712 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:51.732 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:51.732 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:51.753 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:51.753 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:51.758 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:51.754+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:51.820 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:51.820 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:51.835 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:51.859 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:51.853+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:51.960 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:51.957+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:51.999 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:51.999 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:52.061 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:52.057+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:52.144 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:52.145 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:52.160 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:52.158+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:52.226 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone5 from 0xa25e05b (overlap 0x3e5d750) to 0x0 2024-09-12T09:29:52.262 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:52.257+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:52.300 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:52.300 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:52.363 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:52.362+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:52.456 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.462 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:52.457+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:52.473 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:52.563 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:52.561+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:52.659 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:52.659 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.664 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:52.662+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:52.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:52.696 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-12T09:29:52.765 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:52.761+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:52.834 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:52.834 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:52.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:52.895 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:52.893+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:52.966 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:52.961+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:53.015 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.015 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:53.075 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:53.073+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:53.116 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:29:53.116 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:29:53.116 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-12T09:29:53.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-12T09:29:53.150 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.150 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:29:53.164 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:53.204 INFO:teuthology.orchestra.run.smithi007.stdout:766 trunc from 0xa25e05b to 0x6475cdf 2024-09-12T09:29:53.204 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:53.201+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:53.232 INFO:teuthology.orchestra.run.smithi007.stdout:768 read 0x3368fe6 thru 0x336b64e (0x2669 bytes) 2024-09-12T09:29:53.233 INFO:teuthology.orchestra.run.smithi007.stdout:769 write 0x1a1512b thru 0x1a19940 (0x4816 bytes) 2024-09-12T09:29:53.268 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:53.265+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:53.352 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.352 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.368 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:53.369 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:53.365+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:53.389 INFO:teuthology.orchestra.run.smithi007.stdout:770 write 0x2d3080e thru 0x2d3fd3e (0xf531 bytes) 2024-09-12T09:29:53.454 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:29:53.449+0000 7f3454e55640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-12T09:29:53.470 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:53.465+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:53.505 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.505 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:53.571 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:53.565+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:53.577 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:53.593 INFO:teuthology.orchestra.run.smithi007.stdout:773 write 0xa65e859 thru 0xa661230 (0x29d8 bytes) 2024-09-12T09:29:53.600 INFO:teuthology.orchestra.run.smithi007.stdout:775 punch from 0x777a2b4 to 0x777cbfe, (0x294a bytes) 2024-09-12T09:29:53.600 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:53.610 INFO:teuthology.orchestra.run.smithi007.stdout:777 write 0xbd25ee8 thru 0xbd346dd (0xe7f6 bytes) 2024-09-12T09:29:53.621 INFO:teuthology.orchestra.run.smithi007.stdout:778 read 0x7492d11 thru 0x749af04 (0x81f4 bytes) 2024-09-12T09:29:53.621 INFO:teuthology.orchestra.run.smithi007.stdout:779 clone 7 order 25 su 262144 sc 15 2024-09-12T09:29:53.672 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:53.669+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:53.685 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.685 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:53.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:53.773 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:53.769+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:53.833 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:53.833 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:53.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:53.874 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:53.869+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:53.919 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:53.936 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:53.953 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:53.953 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:53.970 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:53.974 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:53.969+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:53.978 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:53.978 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:53.978 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:53.978 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:53.978 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:53.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:53.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:54.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:54.013 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:54.034 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:54.073 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:54.073 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:54.076 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:54.073+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.0. 2024-09-12T09:29:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:29:54.149 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-12T09:29:54.168 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2919 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 942080~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5920878429999998, 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041365445000000001, 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2919 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575879+0000", 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575880+0000", 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7500000000000001e-07 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575881+0000", 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575884+0000", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4060000000000001e-06 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575887+0000", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7930000000000002e-06 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616841+0000", 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040954162000000002 2024-09-12T09:29:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.616963+0000", 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012160800000000001 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617245+0000", 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028208299999999998 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2920 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 974848~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5920763099999995, 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041906318999999997, 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2920 2024-09-12T09:29:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575892+0000", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0950000000000001e-06 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575895+0000", 2024-09-12T09:29:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575902+0000", 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4489999999999996e-06 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617280+0000", 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041377835000000002 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617372+0000", 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2070000000000004e-05 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617797+0000", 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000425284 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2921 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1007616~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5920596069999995, 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042305425000000001, 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2921 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575909+0000", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429999999999999e-06 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575912+0000", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5659999999999998e-06 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575915+0000", 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8880000000000001e-06 2024-09-12T09:29:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617854+0000", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041939002000000003 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617933+0000", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8893000000000005e-05 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618213+0000", 2024-09-12T09:29:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000279671 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2922 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1040384~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5920475869999997, 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042845688999999999, 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2922 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575921+0000", 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0139999999999999e-06 2024-09-12T09:29:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575924+0000", 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3330000000000001e-06 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575927+0000", 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6469999999999998e-06 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618438+0000", 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042511372999999998 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618552+0000", 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114024 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618765+0000", 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213069 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5493 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4192324599999999, 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041875908000000003, 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5493 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748736+0000", 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1939999999999999e-06 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748746+0000", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5969999999999998e-06 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748752+0000", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790107+0000", 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041354634000000001 2024-09-12T09:29:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790210+0000", 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010341899999999999 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790611+0000", 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000400212 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5494 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4192068429999996, 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042226430000000002, 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5494 2024-09-12T09:29:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748761+0000", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748762+0000", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.739e-06 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748767+0000", 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1609999999999996e-06 2024-09-12T09:29:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748770+0000", 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9500000000000001e-06 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790647+0000", 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041877437000000003 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790753+0000", 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105726 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790987+0000", 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023401800000000001 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5495 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4191819799999994, 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042594770999999997, 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5495 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748787+0000", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2810000000000001e-06 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748796+0000", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748801+0000", 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3850000000000001e-06 2024-09-12T09:29:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791018+0000", 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042216562999999999 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791093+0000", 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5080999999999995e-05 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791380+0000", 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028708999999999999 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5496 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2744320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4191613269999994, 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042916190999999999, 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5496 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.35e-07 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748807+0000", 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.237e-06 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748810+0000", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2210000000000002e-06 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748813+0000", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5959999999999998e-06 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791412+0000", 2024-09-12T09:29:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042598534 2024-09-12T09:29:54.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791488+0000", 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5994000000000005e-05 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791722+0000", 2024-09-12T09:29:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234374 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5497 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4191501510000002, 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043287796000000003, 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5497 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748818+0000", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.113e-06 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748828+0000", 2024-09-12T09:29:54.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9019999999999994e-06 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748835+0000", 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4400000000000002e-06 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791754+0000", 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042919481000000002 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791836+0000", 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1514000000000006e-05 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792105+0000", 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000269125 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.213 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5498 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4191167579999995, 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043631883000000003, 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5498 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748851+0000", 2024-09-12T09:29:54.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748852+0000", 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7680000000000001e-06 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748857+0000", 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5510000000000001e-06 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748860+0000", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.157e-06 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792142+0000", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043281687999999999 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792246+0000", 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103749 2024-09-12T09:29:54.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792482+0000", 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023665099999999999 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5499 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2842624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.419102509, 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043996252, 2024-09-12T09:29:54.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5499 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748885+0000", 2024-09-12T09:29:54.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0276999999999999e-05 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748894+0000", 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7809999999999994e-06 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748898+0000", 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6669999999999998e-06 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792530+0000", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043632399000000002 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792612+0000", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2317999999999993e-05 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792861+0000", 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024859200000000001 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5500 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4190628790000002, 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044345271999999998, 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5500 2024-09-12T09:29:54.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748905+0000", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748906+0000", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2300000000000001e-06 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748910+0000", 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2289999999999997e-06 2024-09-12T09:29:54.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748913+0000", 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7439999999999999e-06 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792921+0000", 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044007743000000002 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793009+0000", 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8284999999999999e-05 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793250+0000", 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000240673 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5501 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2908160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:54.227 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4190504110000006, 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044652050999999998, 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5501 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:54.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748918+0000", 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.128e-06 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748921+0000", 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9890000000000001e-06 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748924+0000", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793279+0000", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044355798000000002 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793363+0000", 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3100000000000001e-05 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793569+0000", 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020620799999999999 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5502 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4190398860000002, 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045033795000000001, 2024-09-12T09:29:54.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5502 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:54.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748929+0000", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2610000000000001e-06 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748934+0000", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2340000000000002e-06 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748938+0000", 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2640000000000002e-06 2024-09-12T09:29:54.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793678+0000", 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044739745999999997 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793730+0000", 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909000000000001e-05 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793961+0000", 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023114199999999999 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5503 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.419022279, 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045434384000000001, 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5503 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748947+0000", 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.362e-06 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748951+0000", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5539999999999999e-06 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748954+0000", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794016+0000", 2024-09-12T09:29:54.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045062229000000002 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794103+0000", 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7168000000000003e-05 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794379+0000", 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027592900000000001 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5504 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4190090669999993, 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045942016000000002, 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5504 2024-09-12T09:29:54.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1800000000000002e-07 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748960+0000", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.779e-06 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748966+0000", 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5650000000000001e-06 2024-09-12T09:29:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748971+0000", 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794420+0000", 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045449472999999997 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794513+0000", 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2631000000000001e-05 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794900+0000", 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038728900000000003 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5505 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4189763590000002, 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046773908000000003, 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5505 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8700000000000005e-07 2024-09-12T09:29:54.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748994+0000", 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749020+0000", 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6032000000000001e-05 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749028+0000", 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794934+0000", 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045905796999999998 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795020+0000", 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5636000000000005e-05 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795765+0000", 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074500200000000003 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5506 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4189272430000006, 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047076258000000003, 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5506 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749043+0000", 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7879999999999998e-06 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749057+0000", 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4277000000000001e-05 2024-09-12T09:29:54.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749067+0000", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999999e-06 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795798+0000", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046730657000000002 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795882+0000", 2024-09-12T09:29:54.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4432999999999996e-05 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796116+0000", 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023388599999999999 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5507 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:54.256 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4188901400000002, 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047404593000000002, 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5507 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-12T09:29:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749080+0000", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7159999999999999e-06 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749088+0000", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0250000000000002e-06 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749094+0000", 2024-09-12T09:29:54.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796153+0000", 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047059813999999998 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796244+0000", 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0463000000000004e-05 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796482+0000", 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0002377 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5508 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4188659379999997, 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047813034999999997, 2024-09-12T09:29:54.261 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5508 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:54.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749102+0000", 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749104+0000", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749110+0000", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749116+0000", 2024-09-12T09:29:54.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5889999999999999e-06 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796574+0000", 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047457634999999998 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796658+0000", 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4040000000000002e-05 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796914+0000", 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000256238 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.266 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:54.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:54.268 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:54.265+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:54.369 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:54.365+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:54.442 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:54.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:54.442 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:54.442 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:54.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:54.508 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:54.505+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:54.570 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:54.565+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:54.584 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:54.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:54.672 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:54.669+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:54.771 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:54.771 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:54.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:54.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:54.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.719309688999999, 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:54.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:29:54.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:29:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:29:54.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9987579049999997, 2024-09-12T09:29:54.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:29:54.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:29:54.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:29:54.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9972539129999998, 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:54.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:29:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:29:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9972311640000004, 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:29:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:29:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9972077240000008, 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:29:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:29:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9971842760000005, 2024-09-12T09:29:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:29:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9971408400000001, 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:29:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9971040430000002, 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:29:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9291158030000002, 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9290602430000003, 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9290309939999997, 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9290072899999995, 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:29:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9289875619999997, 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9289640400000003, 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:29:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9289410660000001, 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:29:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:29:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9288968769999997, 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:29:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9288625100000001, 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:29:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8754391409999993, 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8748743050000005, 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:29:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:29:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10623 3.4 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.2485981060000002, 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017782925000000002, 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10623 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520647+0000", 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9369999999999998e-06 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520656+0000", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520672+0000", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6018000000000001e-05 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520680+0000", 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6079999999999998e-06 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520788+0000", 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010822 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520800+0000", 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1753000000000001e-05 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.523800+0000", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030001049999999999 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.524016+0000", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021596300000000001 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527527+0000", 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003511424 2024-09-12T09:29:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538252+0000", 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010724768000000001 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538311+0000", 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9432999999999998e-05 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538380+0000", 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8083000000000005e-05 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538428+0000", 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8906000000000002e-05 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:54.879 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:54.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:54.881 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:54.877+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:54.982 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:54.977+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:55.061 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:55.062 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:55.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:55.120 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:55.117+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:55.184 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:55.181+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:55.272 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:55.272 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:55.285 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:55.285+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:55.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:55.386 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:55.381+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1015 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'4, mlcod=27'2)", 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.987470179000001, 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001427214, 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422637+0000", 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:29:55.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422646+0000", 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7080000000000003e-06 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422678+0000", 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1829000000000003e-05 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422692+0000", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4116e-05 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422811+0000", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011919899999999999 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422852+0000", 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1016000000000002e-05 2024-09-12T09:29:55.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.423982+0000", 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011297950000000001 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.424063+0000", 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1023e-05 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.944626763, 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:29:55.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:29:55.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1039 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'13, mlcod=28'11)", 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.924388411000001, 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013168500000000001, 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485720+0000", 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1670000000000002e-06 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485733+0000", 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3183000000000001e-05 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485765+0000", 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1866000000000003e-05 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485785+0000", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9830000000000002e-05 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485917+0000", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013222599999999999 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485968+0000", 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1106000000000002e-05 2024-09-12T09:29:55.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.486960+0000", 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099206899999999994 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487034+0000", 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4402999999999999e-05 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.879963138000001, 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:29:55.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.875394256, 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:29:55.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.697732844000001, 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:29:55.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.674774479, 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:29:55.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.24442606, 2024-09-12T09:29:55.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:29:55.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:3734 3.4 e33/23 3:21777318:::rbd_data.11912e4e65e9.0000000000000017:head v 33'1, mlcod=0'0)", 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.761218006, 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001459442, 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648890+0000", 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.829e-06 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649057+0000", 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729099999999999 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649076+0000", 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8902e-05 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649083+0000", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.277e-06 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649153+0000", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9380000000000003e-05 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649177+0000", 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.425e-05 2024-09-12T09:29:55.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650298+0000", 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011215369999999999 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650347+0000", 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8976e-05 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:4220 3.4 e33/23 3:245d796b:::rbd_data.11912e4e65e9.0000000000000026:head v 33'2, mlcod=0'0)", 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7283128530000003, 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014342420000000001, 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681795+0000", 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681925+0000", 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012997400000000001 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681944+0000", 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9531e-05 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681954+0000", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5149999999999995e-06 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682029+0000", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5421999999999997e-05 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682056+0000", 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6942000000000001e-05 2024-09-12T09:29:55.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683173+0000", 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001117093 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683227+0000", 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000002e-05 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5962 3.4 e33/23 3:21146361:::rbd_data.11912e4e65e9.000000000000005d:head v 33'3, mlcod=33'1)", 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.6270433549999996, 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001433824, 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:55.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783064+0000", 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.798e-06 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783189+0000", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012434600000000001 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783217+0000", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8218e-05 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783230+0000", 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2965e-05 2024-09-12T09:29:55.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783343+0000", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113309 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783366+0000", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2625000000000001e-05 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784454+0000", 2024-09-12T09:29:55.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001088356 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784496+0000", 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2206999999999998e-05 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5964 3.0 e33/23 3:02a4fae5:::rbd_data.11912e4e65e9.000000000000005f:head v 33'32, mlcod=31'29)", 2024-09-12T09:29:55.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.6191665059999991, 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001299197, 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790941+0000", 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5510000000000001e-06 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791013+0000", 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2242000000000004e-05 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791043+0000", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0028000000000001e-05 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791053+0000", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0154e-05 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791195+0000", 2024-09-12T09:29:55.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014127599999999999 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791216+0000", 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-05 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792186+0000", 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096983200000000001 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792239+0000", 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2814e-05 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.6156527389999997, 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:29:55.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:29:55.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.6148771069999999, 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:29:55.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:29:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2789609970000004, 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:29:55.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.27215232, 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:29:55.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:29:55.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8832047540000003, 2024-09-12T09:29:55.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:29:55.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:29:55.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8798347209999999, 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:29:55.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3742149269999997, 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:29:55.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:29:55.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11694 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'17, mlcod=38'15)", 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:29:55.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3709274330000003, 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015297240000000001, 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:29:55.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039276+0000", 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6244999999999998e-05 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039292+0000", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039302+0000", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9369999999999999e-06 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039368+0000", 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5711999999999997e-05 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039387+0000", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9386999999999999e-05 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040592+0000", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012047270000000001 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040708+0000", 2024-09-12T09:29:55.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116242 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:55.507 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:55.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:29:55.509 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:55.505+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:55.609 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:55.609+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:55.681 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:55.682 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:55.682 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:55.682 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:55.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:29:55.744 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:55.741+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:55.811 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:55.809+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:55.822 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:55.822 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:55.822 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:55.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:29:55.924 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:55.921+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:56.012 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10970 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9076736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2973264530000002, 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032392929000000001, 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10970 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712052+0000", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1699999999999998e-07 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712055+0000", 2024-09-12T09:29:56.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712066+0000", 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1266e-05 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712090+0000", 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3816e-05 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744359+0000", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032268968000000002 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744396+0000", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7233000000000001e-05 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744444+0000", 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8115999999999998e-05 2024-09-12T09:29:56.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10975 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9404416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2967415769999997, 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032398173000000002, 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10975 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712637+0000", 2024-09-12T09:29:56.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6899999999999997e-07 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712640+0000", 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712650+0000", 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189e-05 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712657+0000", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3699999999999997e-06 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744949+0000", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032291804 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744986+0000", 2024-09-12T09:29:56.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7211e-05 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745034+0000", 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7947e-05 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10976 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9469952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:29:56.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.296693501, 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032464923999999999, 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10976 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712685+0000", 2024-09-12T09:29:56.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712687+0000", 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8509999999999999e-06 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712695+0000", 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712699+0000", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5059999999999997e-06 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745063+0000", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032363839999999998 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745101+0000", 2024-09-12T09:29:56.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7833999999999997e-05 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745149+0000", 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8288999999999998e-05 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10977 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9535488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.296671914, 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032562971000000003, 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10977 2024-09-12T09:29:56.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8500000000000002e-07 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712716+0000", 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5070000000000006e-06 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712721+0000", 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5079999999999996e-06 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712726+0000", 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8919999999999999e-06 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745178+0000", 2024-09-12T09:29:56.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032451498000000002 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745216+0000", 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7973999999999999e-05 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745269+0000", 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3106999999999998e-05 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10978 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9601024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2966454929999998, 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032657185999999998, 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10978 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7099999999999997e-07 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712734+0000", 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5120000000000001e-06 2024-09-12T09:29:56.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712741+0000", 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8040000000000004e-06 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712745+0000", 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3470000000000001e-06 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745298+0000", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032552400000000002 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745336+0000", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8191000000000001e-05 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745389+0000", 2024-09-12T09:29:56.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3560999999999998e-05 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10979 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9666560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2966257160000003, 2024-09-12T09:29:56.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032757778000000001, 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10979 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-12T09:29:56.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712754+0000", 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5910000000000001e-06 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712759+0000", 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9960000000000001e-06 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712764+0000", 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745418+0000", 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032653765000000001 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745456+0000", 2024-09-12T09:29:56.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8197e-05 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745510+0000", 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3344999999999998e-05 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10980 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9732096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:29:56.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2966079769999999, 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032858152000000002, 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10980 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-12T09:29:56.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712772+0000", 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712776+0000", 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7079999999999996e-06 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712780+0000", 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0890000000000002e-06 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745539+0000", 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032758879999999997 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745577+0000", 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7704999999999999e-05 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745628+0000", 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0850000000000003e-05 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10981 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9797632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965918810000003, 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974687000000003, 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10981 2024-09-12T09:29:56.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712788+0000", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712792+0000", 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5299999999999998e-06 2024-09-12T09:29:56.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712796+0000", 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9199999999999997e-06 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745657+0000", 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032860633 2024-09-12T09:29:56.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745701+0000", 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4546999999999999e-05 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745760+0000", 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9228000000000003e-05 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10982 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9863168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965747409999997, 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033077600999999998, 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10982 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1300000000000001e-07 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712805+0000", 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:29:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712811+0000", 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712815+0000", 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9720000000000003e-06 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745790+0000", 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974742000000001 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745828+0000", 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7870000000000002e-05 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745880+0000", 2024-09-12T09:29:56.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2512999999999997e-05 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10983 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9928704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965565740000002, 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033178996000000002, 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10983 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2899999999999999e-07 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712823+0000", 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.522e-06 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712827+0000", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6e-06 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712831+0000", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8580000000000002e-06 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745910+0000", 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033078308000000001 2024-09-12T09:29:56.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745948+0000", 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8012000000000001e-05 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746000+0000", 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2367000000000002e-05 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10984 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9994240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965410779999997, 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033283132, 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10984 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712837+0000", 2024-09-12T09:29:56.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2599999999999998e-07 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712838+0000", 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.505e-06 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712843+0000", 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712847+0000", 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746028+0000", 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033181575999999997 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746066+0000", 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7892000000000002e-05 2024-09-12T09:29:56.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746120+0000", 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3375999999999999e-05 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10985 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10059776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:29:56.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965256150000002, 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033383775999999997, 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10985 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2399999999999999e-07 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712854+0000", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712859+0000", 2024-09-12T09:29:56.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712862+0000", 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8130000000000002e-06 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746148+0000", 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033285631000000003 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746185+0000", 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7381e-05 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746236+0000", 2024-09-12T09:29:56.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0334999999999998e-05 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10986 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10125312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2965101299999997, 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033485506999999998, 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10986 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712868+0000", 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712869+0000", 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.598e-06 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712876+0000", 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.657e-06 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712883+0000", 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4050000000000003e-06 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746264+0000", 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033380552000000001 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746301+0000", 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7187000000000002e-05 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746353+0000", 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1745e-05 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10987 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10190848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2964824149999998, 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033575365000000003, 2024-09-12T09:29:56.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10987 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:29:56.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712896+0000", 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4799999999999998e-07 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712899+0000", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9409999999999999e-06 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712910+0000", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712918+0000", 2024-09-12T09:29:56.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8709999999999997e-06 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746381+0000", 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033463443000000002 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746419+0000", 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7415e-05 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746471+0000", 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1903000000000001e-05 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10988 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10256384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:29:56.088 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2964497069999998, 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033660336999999999, 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10988 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712931+0000", 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6309999999999999e-06 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712941+0000", 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7149999999999993e-06 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712948+0000", 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6129999999999998e-06 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746499+0000", 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033550330000000003 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746536+0000", 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7413000000000002e-05 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746588+0000", 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2154999999999998e-05 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10989 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10321920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2964194110000005, 2024-09-12T09:29:56.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033748256999999997, 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10989 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712959+0000", 2024-09-12T09:29:56.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9100000000000004e-07 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712967+0000", 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1839999999999999e-06 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712988+0000", 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1356999999999999e-05 2024-09-12T09:29:56.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712997+0000", 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746617+0000", 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033619818000000003 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746654+0000", 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7419000000000002e-05 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746706+0000", 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2318000000000003e-05 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10990 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10387456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2963679890000002, 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033815079999999997, 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10990 2024-09-12T09:29:56.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:29:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8899999999999999e-07 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713015+0000", 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2950000000000004e-06 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713025+0000", 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713033+0000", 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746735+0000", 2024-09-12T09:29:56.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033701704999999998 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746772+0000", 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7573000000000001e-05 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746825+0000", 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2268999999999997e-05 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10991 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10452992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2963338220000002, 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033896353999999997, 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10991 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713549+0000", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050486900000000002 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713554+0000", 2024-09-12T09:29:56.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8660000000000002e-06 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713569+0000", 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4497e-05 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713579+0000", 2024-09-12T09:29:56.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0074e-05 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746853+0000", 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033273529000000003 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746889+0000", 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746940+0000", 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0640999999999998e-05 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10992 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10518528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2957809109999996, 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033468748999999999, 2024-09-12T09:29:56.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10992 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:29:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713601+0000", 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713612+0000", 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0429e-05 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713619+0000", 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000001e-06 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746969+0000", 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033349733999999999 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747007+0000", 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7576000000000001e-05 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747065+0000", 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8770999999999997e-05 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10993 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10584064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:29:56.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2957470359999999, 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033553980999999997, 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10993 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720607+0000", 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069764909999999996 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720616+0000", 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2289999999999992e-06 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:56.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720631+0000", 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4483e-05 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720640+0000", 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3749999999999992e-06 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747095+0000", 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026454688000000001 2024-09-12T09:29:56.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747133+0000", 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7719000000000003e-05 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747185+0000", 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1996000000000002e-05 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:56.123 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:56.123 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:56.009+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:56.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:29:56.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:29:56.126 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:56.121+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:56.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:29:56.227 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:56.221+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:29:56.256 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:29:56.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:29:56.320 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:56.320 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:56.328 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:56.325+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:56.337 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:56.337 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:56.356 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:56.356 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:56.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:56.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:56.391 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:56.392 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:56.393 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:56.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:29:56.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:56.431 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:56.431 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:56.451 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:56.449+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:56.457 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:56.468 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone6 from 0xbd346de (overlap 0x6475cdf) to 0xa0bc7c 2024-09-12T09:29:56.529 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:56.525+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:56.630 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:56.630 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:56.630 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:56.625+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:56.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:29:56.731 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:56.729+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:56.820 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:56.820 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:56.833 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:56.829+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:56.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:29:56.934 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:56.929+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:56.995 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:56.995 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:29:57.035 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:57.033+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:29:57.137 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:57.133+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:57.220 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:57.220 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.234 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:29:57.237 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:57.233+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:57.338 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:29:57.333+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:29:57.367 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:57.368 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:29:57.439 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:57.437+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:57.514 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:57.514 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:29:57.575 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:57.573+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:57.641 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:57.637+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:57.673 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:57.673 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:29:57.742 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:57.737+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:57.843 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:57.841+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:57.889 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:57.889 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:57.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:29:57.951 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:57.949+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:58.045 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:58.041+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:58.057 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:58.057 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:58.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:29:58.145 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:58.141+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:58.196 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:58.196 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:58.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:29:58.255 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:58.253+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:58.347 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:58.341+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:58.356 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:58.356 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:58.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:29:58.457 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-12T09:29:58.458 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:58.453+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:58.461 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:58.461 INFO:teuthology.orchestra.run.smithi007.stdout:781 read 0x72402a4 thru 0x7242e54 (0x2bb1 bytes) 2024-09-12T09:29:58.461 INFO:teuthology.orchestra.run.smithi007.stdout:782 write 0x350ae8a thru 0x350b0ad (0x224 bytes) 2024-09-12T09:29:58.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:29:58.549 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:58.545+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:58.570 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:29:58.570 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:29:58.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:29:58.600 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:58.600 INFO:teuthology.orchestra.run.smithi007.stdout:784 write 0x190a497 thru 0x190c46d (0x1fd7 bytes) 2024-09-12T09:29:58.649 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:58.645+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:58.653 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:29:58.653 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:29:58.673 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:29:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:29:58.690 INFO:teuthology.orchestra.run.smithi007.stdout:785 write 0xc0fbb31 thru 0xc101ee9 (0x63b9 bytes) 2024-09-12T09:29:58.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:29:58.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:29:58.739 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2920 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 974848~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:58.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.173530814999999, 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041906318999999997, 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2920 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575891+0000", 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575892+0000", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0950000000000001e-06 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575895+0000", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575902+0000", 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4489999999999996e-06 2024-09-12T09:29:58.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617280+0000", 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041377835000000002 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617372+0000", 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2070000000000004e-05 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617797+0000", 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000425284 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2921 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1007616~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.173514111999999, 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042305425000000001, 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2921 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575908+0000", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575909+0000", 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429999999999999e-06 2024-09-12T09:29:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575912+0000", 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5659999999999998e-06 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575915+0000", 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8880000000000001e-06 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617854+0000", 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041939002000000003 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.617933+0000", 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8893000000000005e-05 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618213+0000", 2024-09-12T09:29:58.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000279671 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:2922 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 1040384~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.173502092, 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042845688999999999, 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2922 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:58.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575920+0000", 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575921+0000", 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0139999999999999e-06 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575924+0000", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3330000000000001e-06 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.575927+0000", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6469999999999998e-06 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618438+0000", 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042511372999999998 2024-09-12T09:29:58.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618552+0000", 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114024 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.618765+0000", 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213069 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5493 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000686965, 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041875908000000003, 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5493 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748735+0000", 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748736+0000", 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1939999999999999e-06 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748746+0000", 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5969999999999998e-06 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748752+0000", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790107+0000", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041354634000000001 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790210+0000", 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010341899999999999 2024-09-12T09:29:58.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790611+0000", 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000400212 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5494 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2678784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000661348, 2024-09-12T09:29:58.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042226430000000002, 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5494 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748760+0000", 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748761+0000", 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748762+0000", 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.739e-06 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748767+0000", 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1609999999999996e-06 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748770+0000", 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9500000000000001e-06 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790647+0000", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041877437000000003 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790753+0000", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105726 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790987+0000", 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023401800000000001 2024-09-12T09:29:58.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5495 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000636484999999, 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042594770999999997, 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5495 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:58.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748785+0000", 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748787+0000", 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2810000000000001e-06 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748796+0000", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1090000000000004e-06 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748801+0000", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3850000000000001e-06 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791018+0000", 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042216562999999999 2024-09-12T09:29:58.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791093+0000", 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5080999999999995e-05 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791380+0000", 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028708999999999999 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5496 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2744320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000615831999999, 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042916190999999999, 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5496 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748806+0000", 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.35e-07 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748807+0000", 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.237e-06 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748810+0000", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2210000000000002e-06 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748813+0000", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5959999999999998e-06 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791412+0000", 2024-09-12T09:29:58.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042598534 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791488+0000", 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5994000000000005e-05 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791722+0000", 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234374 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5497 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2777088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000604656, 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043287796000000003, 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5497 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748817+0000", 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748818+0000", 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.113e-06 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748828+0000", 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9019999999999994e-06 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748835+0000", 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4400000000000002e-06 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791754+0000", 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.042919481000000002 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791836+0000", 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1514000000000006e-05 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792105+0000", 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000269125 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5498 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000571262999999, 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043631883000000003, 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5498 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.790 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748850+0000", 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748851+0000", 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748852+0000", 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7680000000000001e-06 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748857+0000", 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5510000000000001e-06 2024-09-12T09:29:58.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748860+0000", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.157e-06 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792142+0000", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043281687999999999 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792246+0000", 2024-09-12T09:29:58.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103749 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792482+0000", 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023665099999999999 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5499 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2842624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000557014, 2024-09-12T09:29:58.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043996252, 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5499 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:58.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748865+0000", 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748885+0000", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0276999999999999e-05 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748894+0000", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7809999999999994e-06 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748898+0000", 2024-09-12T09:29:58.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6669999999999998e-06 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792530+0000", 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.043632399000000002 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792612+0000", 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2317999999999993e-05 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792861+0000", 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024859200000000001 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5500 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000517384, 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044345271999999998, 2024-09-12T09:29:58.799 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5500 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748904+0000", 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748905+0000", 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6800000000000001e-07 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748906+0000", 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2300000000000001e-06 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748910+0000", 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2289999999999997e-06 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748913+0000", 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7439999999999999e-06 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792921+0000", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044007743000000002 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793009+0000", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8284999999999999e-05 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793250+0000", 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000240673 2024-09-12T09:29:58.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5501 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2908160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000504916000001, 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044652050999999998, 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5501 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748917+0000", 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748918+0000", 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.128e-06 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748921+0000", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9890000000000001e-06 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748924+0000", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793279+0000", 2024-09-12T09:29:58.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044355798000000002 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793363+0000", 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3100000000000001e-05 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793569+0000", 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020620799999999999 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5502 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2940928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000494391, 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045033795000000001, 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5502 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748927+0000", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748929+0000", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2610000000000001e-06 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748934+0000", 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2340000000000002e-06 2024-09-12T09:29:58.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748938+0000", 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2640000000000002e-06 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793678+0000", 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.044739745999999997 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793730+0000", 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909000000000001e-05 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.793961+0000", 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023114199999999999 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5503 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000476784, 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045434384000000001, 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5503 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748945+0000", 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748947+0000", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.362e-06 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748951+0000", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5539999999999999e-06 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748954+0000", 2024-09-12T09:29:58.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794016+0000", 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045062229000000002 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794103+0000", 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7168000000000003e-05 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794379+0000", 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027592900000000001 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5504 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000463571999999, 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045942016000000002, 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5504 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:58.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748958+0000", 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1800000000000002e-07 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748960+0000", 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.779e-06 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748966+0000", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5650000000000001e-06 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748971+0000", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794420+0000", 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045449472999999997 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794513+0000", 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2631000000000001e-05 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794900+0000", 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038728900000000003 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5505 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000430864, 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046773908000000003, 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5505 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748991+0000", 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8700000000000005e-07 2024-09-12T09:29:58.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.748994+0000", 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749020+0000", 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6032000000000001e-05 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749028+0000", 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794934+0000", 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.045905796999999998 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795020+0000", 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5636000000000005e-05 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795765+0000", 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074500200000000003 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5506 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000381748000001, 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047076258000000003, 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5506 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749040+0000", 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749043+0000", 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7879999999999998e-06 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749057+0000", 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4277000000000001e-05 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749067+0000", 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7399999999999999e-06 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795798+0000", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046730657000000002 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795882+0000", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4432999999999996e-05 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796116+0000", 2024-09-12T09:29:58.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023388599999999999 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5507 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000344645, 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047404593000000002, 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5507 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749077+0000", 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749080+0000", 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7159999999999999e-06 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749088+0000", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0250000000000002e-06 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749094+0000", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796153+0000", 2024-09-12T09:29:58.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047059813999999998 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796244+0000", 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0463000000000004e-05 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796482+0000", 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0002377 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5508 3.1f 3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.000320443, 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047813034999999997, 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5508 2024-09-12T09:29:58.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749101+0000", 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749102+0000", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749104+0000", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749110+0000", 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-12T09:29:58.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.749116+0000", 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5889999999999999e-06 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796574+0000", 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047457634999999998 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796658+0000", 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4040000000000002e-05 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796914+0000", 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000256238 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:29:58.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6122380009999997, 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:29:58.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:29:58.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:29:58.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:58.848 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:58.849 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:58.745+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:58.849 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:29:58.849 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:29:58.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:29:58.851 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:29:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:29:58.851 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:29:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:29:58.852 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:29:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:29:58.852 INFO:teuthology.orchestra.run.smithi007.stdout:786 write 0x1f6432d thru 0x1f6f16b (0xae3f bytes) 2024-09-12T09:29:58.892 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:58.889+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:58.955 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:58.949+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:58.966 INFO:teuthology.orchestra.run.smithi007.stdout:787 write 0xd75b794 thru 0xd768b15 (0xd382 bytes) 2024-09-12T09:29:58.969 INFO:teuthology.orchestra.run.smithi007.stdout:788 punch from 0xc917419 to 0xc920ea2, (0x9a89 bytes) 2024-09-12T09:29:58.969 INFO:teuthology.orchestra.run.smithi007.stdout:789 read 0x4aa802d thru 0x4ab58b5 (0xd889 bytes) 2024-09-12T09:29:58.969 INFO:teuthology.orchestra.run.smithi007.stdout:791 read 0xb9acbb5 thru 0xb9b87ce (0xbc1a bytes) 2024-09-12T09:29:58.970 INFO:teuthology.orchestra.run.smithi007.stdout:792 read 0xacc3c63 thru 0xaccb114 (0x74b2 bytes) 2024-09-12T09:29:58.983 INFO:teuthology.orchestra.run.smithi007.stdout:793 trunc from 0xd768b16 to 0x5b60ed7 2024-09-12T09:29:59.048 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:59.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:59.056 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:29:59.053+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:29:59.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:29:59.149 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:29:59.150 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:29:59.150 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-12T09:29:59.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-12T09:29:59.158 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:59.153+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:59.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:29:59.257 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:29:59.253+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:29:59.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:59.296 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:59.296 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:59.296 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:59.296 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:59.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:29:59.372 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:59.369+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:59.459 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:29:59.457+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:29:59.473 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.422663457999999, 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:29:59.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.702111673999999, 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:29:59.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:59.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700607681999999, 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:29:59.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700584933, 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700561493, 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:29:59.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700538045, 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:29:59.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700494609, 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:29:59.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.700457812, 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:29:59.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:29:59.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632469572, 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:29:59.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632414012, 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632384762999999, 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:29:59.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632361059000001, 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:29:59.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632341330999999, 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632317809, 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:29:59.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:29:59.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:29:59.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:29:59.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632294835, 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:29:59.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632250645999999, 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:29:59.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:29:59.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:29:59.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.632216279, 2024-09-12T09:29:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:59.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:29:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.578792910000001, 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:29:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.578228074, 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:29:59.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10623 3.4 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9519518750000007, 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017782925000000002, 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10623 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:29:59.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520647+0000", 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9369999999999998e-06 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520656+0000", 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520672+0000", 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6018000000000001e-05 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520680+0000", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6079999999999998e-06 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520788+0000", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010822 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520800+0000", 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1753000000000001e-05 2024-09-12T09:29:59.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.523800+0000", 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030001049999999999 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.524016+0000", 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021596300000000001 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:29:59.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527527+0000", 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003511424 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538252+0000", 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010724768000000001 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538311+0000", 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9432999999999998e-05 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538380+0000", 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8083000000000005e-05 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538428+0000", 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8906000000000002e-05 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:29:59.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:29:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:59.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:29:59.576 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:59.569+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:59.673 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:29:59.669+0000 7f3454e55640 -1 mon.a@0(leader).osd e45 definitely_dead 0 2024-09-12T09:29:59.675 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:29:59.669+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:29:59.745 INFO:teuthology.orchestra.run.smithi007.stdout:794 write 0x4775e7 thru 0x4814e6 (0x9f00 bytes) 2024-09-12T09:29:59.748 INFO:teuthology.orchestra.run.smithi007.stdout:796 punch from 0x4892fd6 to 0x489ca42, (0x9a6c bytes) 2024-09-12T09:29:59.748 INFO:teuthology.orchestra.run.smithi007.stdout:797 read 0x578003c thru 0x5780f59 (0xf1e bytes) 2024-09-12T09:29:59.749 INFO:teuthology.orchestra.run.smithi007.stdout:798 punch from 0x2126c0c to 0x21307da, (0x9bce bytes) 2024-09-12T09:29:59.749 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:29:59.749 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:59.759 INFO:teuthology.orchestra.run.smithi007.stdout:799 trunc from 0x5b60ed7 to 0xe807521 2024-09-12T09:29:59.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:29:59.770 INFO:teuthology.orchestra.run.smithi007.stdout:800 read 0x192926a thru 0x192be71 (0x2c08 bytes) 2024-09-12T09:29:59.776 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:29:59.773+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:29:59.777 INFO:teuthology.orchestra.run.smithi007.stdout:801 trunc from 0xe807521 to 0x906837e 2024-09-12T09:29:59.790 INFO:teuthology.orchestra.run.smithi007.stdout:802 read 0x5ed9878 thru 0x5ede867 (0x4ff0 bytes) 2024-09-12T09:29:59.794 INFO:teuthology.orchestra.run.smithi007.stdout:803 trunc from 0x906837e to 0x6815b10 2024-09-12T09:29:59.808 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:59.809 INFO:teuthology.orchestra.run.smithi007.stdout:805 flatten 2024-09-12T09:29:59.839 INFO:teuthology.orchestra.run.smithi007.stdout:806 write 0xe37f439 thru 0xe387aa3 (0x866b bytes) 2024-09-12T09:29:59.843 INFO:teuthology.orchestra.run.smithi007.stdout:807 write 0xcbe7788 thru 0xcbf2b90 (0xb409 bytes) 2024-09-12T09:29:59.846 INFO:teuthology.orchestra.run.smithi007.stdout:808 trunc from 0xe387aa4 to 0xdf2a369 2024-09-12T09:29:59.864 INFO:teuthology.orchestra.run.smithi007.stdout:810 punch from 0xba908e2 to 0xbaa0490, (0xfbae bytes) 2024-09-12T09:29:59.864 INFO:teuthology.orchestra.run.smithi007.stdout:812 write 0xaf1c4b3 thru 0xaf1f5d8 (0x3126 bytes) 2024-09-12T09:29:59.868 INFO:teuthology.orchestra.run.smithi007.stdout:814 punch from 0xb823f33 to 0xb831d2c, (0xddf9 bytes) 2024-09-12T09:29:59.868 INFO:teuthology.orchestra.run.smithi007.stdout:815 writesame 0x3342766 thru 0x33524a7 data_size 0x16(0xfd42 bytes) 2024-09-12T09:29:59.874 INFO:teuthology.orchestra.run.smithi007.stdout:816 read 0xba1ecb0 thru 0xba26903 (0x7c54 bytes) 2024-09-12T09:29:59.875 INFO:teuthology.orchestra.run.smithi007.stdout:817 read 0x8eddac0 thru 0x8ee3580 (0x5ac1 bytes) 2024-09-12T09:29:59.876 INFO:teuthology.orchestra.run.smithi007.stdout:818 trunc from 0xdf2a369 to 0xd85d3ae 2024-09-12T09:29:59.877 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:29:59.873+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:29:59.895 INFO:teuthology.orchestra.run.smithi007.stdout:821 write 0xa1dcf0f thru 0xa1e1994 (0x4a86 bytes) 2024-09-12T09:29:59.899 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:29:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:29:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:29:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:29:59.900 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:29:59.900 INFO:teuthology.orchestra.run.smithi007.stdout:823 write 0x212e332 thru 0x213d8e7 (0xf5b6 bytes) 2024-09-12T09:29:59.904 INFO:teuthology.orchestra.run.smithi007.stdout:824 punch from 0x3046a35 to 0x304b664, (0x4c2f bytes) 2024-09-12T09:29:59.904 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:59.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:29:59.919 INFO:teuthology.orchestra.run.smithi007.stdout:828 write 0xec5a806 thru 0xec6592c (0xb127 bytes) 2024-09-12T09:29:59.923 INFO:teuthology.orchestra.run.smithi007.stdout:829 read 0xe194060 thru 0xe1989f5 (0x4996 bytes) 2024-09-12T09:29:59.924 INFO:teuthology.orchestra.run.smithi007.stdout:830 writesame 0x326c009 thru 0x327ad95 data_size 0x57(0xed8d bytes) 2024-09-12T09:29:59.927 INFO:teuthology.orchestra.run.smithi007.stdout:832 read 0x76b05ba thru 0x76be6f7 (0xe13e bytes) 2024-09-12T09:29:59.928 INFO:teuthology.orchestra.run.smithi007.stdout:833 writesame 0xd8a6ef0 thru 0xd8ae174 data_size 0x1(0x7285 bytes) 2024-09-12T09:29:59.946 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:29:59.946 INFO:teuthology.orchestra.run.smithi007.stdout:835 write 0x24a364c thru 0x24a4b8a (0x153f bytes) 2024-09-12T09:29:59.948 INFO:teuthology.orchestra.run.smithi007.stdout:836 writesame 0x6e4c472 thru 0x6e4ef20 data_size 0x7(0x2aaf bytes) 2024-09-12T09:29:59.951 INFO:teuthology.orchestra.run.smithi007.stdout:838 write 0xd136fb1 thru 0xd13b323 (0x4373 bytes) 2024-09-12T09:29:59.954 INFO:teuthology.orchestra.run.smithi007.stdout:840 write 0x2fabb5c thru 0x2fb7d33 (0xc1d8 bytes) 2024-09-12T09:29:59.965 INFO:teuthology.orchestra.run.smithi007.stdout:841 trunc from 0xec6592d to 0x84d104e 2024-09-12T09:29:59.978 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:29:59.973+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:29:59.985 INFO:teuthology.orchestra.run.smithi007.stdout:842 trunc from 0x84d104e to 0x9971168 2024-09-12T09:29:59.996 INFO:teuthology.orchestra.run.smithi007.stdout:843 trunc from 0x9971168 to 0xa0fa211 2024-09-12T09:30:00.010 INFO:teuthology.orchestra.run.smithi007.stdout:844 trunc from 0xa0fa211 to 0xd76a28a 2024-09-12T09:30:00.023 INFO:teuthology.orchestra.run.smithi007.stdout:845 trunc from 0xd76a28a to 0xb7d63c5 2024-09-12T09:30:00.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1015 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'4, mlcod=27'2)", 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.623532950000001, 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001427214, 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422637+0000", 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422646+0000", 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7080000000000003e-06 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422678+0000", 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1829000000000003e-05 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422692+0000", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4116e-05 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422811+0000", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011919899999999999 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422852+0000", 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1016000000000002e-05 2024-09-12T09:30:00.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.423982+0000", 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011297950000000001 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.424063+0000", 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1023e-05 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.580689534000001, 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:30:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:30:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1039 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'13, mlcod=28'11)", 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.560451182000001, 2024-09-12T09:30:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013168500000000001, 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485720+0000", 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1670000000000002e-06 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485733+0000", 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3183000000000001e-05 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485765+0000", 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1866000000000003e-05 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485785+0000", 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9830000000000002e-05 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485917+0000", 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013222599999999999 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485968+0000", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1106000000000002e-05 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.486960+0000", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099206899999999994 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487034+0000", 2024-09-12T09:30:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4402999999999999e-05 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.516025909, 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:30:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.511457026999999, 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:30:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:30:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.333795615, 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:30:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:30:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.310837249999999, 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:30:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:30:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.880488830999999, 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:30:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:3734 3.4 e33/23 3:21777318:::rbd_data.11912e4e65e9.0000000000000017:head v 33'1, mlcod=0'0)", 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.397280777000001, 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001459442, 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648890+0000", 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.829e-06 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649057+0000", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729099999999999 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649076+0000", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8902e-05 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649083+0000", 2024-09-12T09:30:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.277e-06 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649153+0000", 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9380000000000003e-05 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649177+0000", 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.425e-05 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650298+0000", 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011215369999999999 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650347+0000", 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8976e-05 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:4220 3.4 e33/23 3:245d796b:::rbd_data.11912e4e65e9.0000000000000026:head v 33'2, mlcod=0'0)", 2024-09-12T09:30:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.364375624000001, 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014342420000000001, 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681795+0000", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681925+0000", 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012997400000000001 2024-09-12T09:30:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681944+0000", 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9531e-05 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681954+0000", 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5149999999999995e-06 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682029+0000", 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5421999999999997e-05 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682056+0000", 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6942000000000001e-05 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683173+0000", 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001117093 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683227+0000", 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000002e-05 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5962 3.4 e33/23 3:21146361:::rbd_data.11912e4e65e9.000000000000005d:head v 33'3, mlcod=33'1)", 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.263106126, 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001433824, 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783064+0000", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.798e-06 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783189+0000", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012434600000000001 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783217+0000", 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8218e-05 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783230+0000", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2965e-05 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783343+0000", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113309 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783366+0000", 2024-09-12T09:30:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2625000000000001e-05 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784454+0000", 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001088356 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784496+0000", 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2206999999999998e-05 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5964 3.0 e33/23 3:02a4fae5:::rbd_data.11912e4e65e9.000000000000005f:head v 33'32, mlcod=31'29)", 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.255229277, 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001299197, 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790941+0000", 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5510000000000001e-06 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791013+0000", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2242000000000004e-05 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791043+0000", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0028000000000001e-05 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791053+0000", 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0154e-05 2024-09-12T09:30:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791195+0000", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014127599999999999 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791216+0000", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-05 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792186+0000", 2024-09-12T09:30:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096983200000000001 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792239+0000", 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2814e-05 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.25171551, 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:30:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.250939878000001, 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9150237679999993, 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9082150910000006, 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:30:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5192675250000001, 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:30:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5158974920000006, 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:30:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:30:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0102776979999994, 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:30:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:30:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11694 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'17, mlcod=38'15)", 2024-09-12T09:30:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0069902039999992, 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015297240000000001, 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039276+0000", 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6244999999999998e-05 2024-09-12T09:30:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039292+0000", 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039302+0000", 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9369999999999999e-06 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039368+0000", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5711999999999997e-05 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039387+0000", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9386999999999999e-05 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040592+0000", 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012047270000000001 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040708+0000", 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116242 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.143 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:00.144 INFO:teuthology.orchestra.run.smithi007.stdout:846 write 0xd0caa24 thru 0xd0d6500 (0xbadd bytes) 2024-09-12T09:30:00.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:00.146 INFO:teuthology.orchestra.run.smithi007.stdout:847 trunc from 0xd0d6501 to 0x14637a5 2024-09-12T09:30:00.146 INFO:teuthology.orchestra.run.smithi007.stdout:849 trunc from 0x14637a5 to 0xd98ea90 2024-09-12T09:30:00.146 INFO:teuthology.orchestra.run.smithi007.stdout:850 read 0x4aa05f1 thru 0x4aaf7c0 (0xf1d0 bytes) 2024-09-12T09:30:00.146 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xedc5658 2024-09-12T09:30:00.146 INFO:teuthology.orchestra.run.smithi007.stdout:851 trunc from 0xd98ea90 to 0xedc5658 2024-09-12T09:30:00.146 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:00.141+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:00.153 INFO:teuthology.orchestra.run.smithi007.stdout:852 read 0x6ece1cf thru 0x6ed8723 (0xa555 bytes) 2024-09-12T09:30:00.163 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.6. 2024-09-12T09:30:00.167 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:30:00.175 INFO:teuthology.orchestra.run.smithi007.stdout:853 trunc from 0xedc5658 to 0x1a538e4 2024-09-12T09:30:00.190 INFO:teuthology.orchestra.run.smithi007.stdout:854 trunc from 0x1a538e4 to 0xf9cf92 2024-09-12T09:30:00.218 INFO:teuthology.orchestra.run.smithi007.stdout:855 write 0xb86e462 thru 0xb871e54 (0x39f3 bytes) 2024-09-12T09:30:00.226 INFO:teuthology.orchestra.run.smithi007.stdout:856 trunc from 0xb871e55 to 0x5f12815 2024-09-12T09:30:00.244 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:00.241+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:00.251 INFO:teuthology.orchestra.run.smithi007.stdout:857 writesame 0xdc49563 thru 0xdc54703 data_size 0x1(0xb1a1 bytes) 2024-09-12T09:30:00.252 INFO:teuthology.orchestra.run.smithi007.stdout:859 read 0x713863e thru 0x7146d48 (0xe70b bytes) 2024-09-12T09:30:00.253 INFO:teuthology.orchestra.run.smithi007.stdout:860 write 0x89d6492 thru 0x89d9d8d (0x38fc bytes) 2024-09-12T09:30:00.255 INFO:teuthology.orchestra.run.smithi007.stdout:861 write 0x2a414f7 thru 0x2a47a54 (0x655e bytes) 2024-09-12T09:30:00.257 INFO:teuthology.orchestra.run.smithi007.stdout:862 write 0x5fd4bfb thru 0x5fd6b4a (0x1f50 bytes) 2024-09-12T09:30:00.259 INFO:teuthology.orchestra.run.smithi007.stdout:863 read 0xcb88055 thru 0xcb8ba86 (0x3a32 bytes) 2024-09-12T09:30:00.260 INFO:teuthology.orchestra.run.smithi007.stdout:864 writesame 0x17c122f thru 0x17c7fb8 data_size 0x7(0x6d8a bytes) 2024-09-12T09:30:00.263 INFO:teuthology.orchestra.run.smithi007.stdout:867 write 0xc725904 thru 0xc73074b (0xae48 bytes) 2024-09-12T09:30:00.265 INFO:teuthology.orchestra.run.smithi007.stdout:868 write 0x6b4aef0 thru 0x6b4d890 (0x29a1 bytes) 2024-09-12T09:30:00.267 INFO:teuthology.orchestra.run.smithi007.stdout:869 clone 8 order 21 su 262144 sc 7 2024-09-12T09:30:00.287 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:00.287 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:00.287 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:00.287 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:00.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:00.345 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:00.341+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:00.446 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:00.441+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:00.468 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:00.469 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:00.469 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:00.469 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:00.469 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:00.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:00.568 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:00.565+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:00.647 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:00.645+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10970 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9076736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:30:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9349608049999993, 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032392929000000001, 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10970 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712051+0000", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712052+0000", 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1699999999999998e-07 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712055+0000", 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712066+0000", 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1266e-05 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712090+0000", 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3816e-05 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744359+0000", 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032268968000000002 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744396+0000", 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7233000000000001e-05 2024-09-12T09:30:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744444+0000", 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8115999999999998e-05 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10975 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9404416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9343759289999998, 2024-09-12T09:30:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032398173000000002, 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10975 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712637+0000", 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6899999999999997e-07 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712640+0000", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712650+0000", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189e-05 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712657+0000", 2024-09-12T09:30:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3699999999999997e-06 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744949+0000", 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032291804 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744986+0000", 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7211e-05 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745034+0000", 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7947e-05 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10976 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9469952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9343278529999992, 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032464923999999999, 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10976 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712685+0000", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712687+0000", 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8509999999999999e-06 2024-09-12T09:30:00.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712695+0000", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712699+0000", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5059999999999997e-06 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745063+0000", 2024-09-12T09:30:00.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032363839999999998 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745101+0000", 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7833999999999997e-05 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745149+0000", 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8288999999999998e-05 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10977 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9535488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9343062660000001, 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032562971000000003, 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10977 2024-09-12T09:30:00.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8500000000000002e-07 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712716+0000", 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5070000000000006e-06 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712721+0000", 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5079999999999996e-06 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712726+0000", 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8919999999999999e-06 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745178+0000", 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032451498000000002 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745216+0000", 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7973999999999999e-05 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745269+0000", 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3106999999999998e-05 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10978 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9601024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9342798450000007, 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032657185999999998, 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10978 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7099999999999997e-07 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712734+0000", 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5120000000000001e-06 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712741+0000", 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8040000000000004e-06 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712745+0000", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3470000000000001e-06 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745298+0000", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032552400000000002 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745336+0000", 2024-09-12T09:30:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8191000000000001e-05 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745389+0000", 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3560999999999998e-05 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10979 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9666560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9342600680000004, 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032757778000000001, 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10979 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-12T09:30:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712754+0000", 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5910000000000001e-06 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712759+0000", 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9960000000000001e-06 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712764+0000", 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745418+0000", 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032653765000000001 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745456+0000", 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8197e-05 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745510+0000", 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3344999999999998e-05 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10980 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9732096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9342423289999999, 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032858152000000002, 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10980 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712772+0000", 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712776+0000", 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7079999999999996e-06 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712780+0000", 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0890000000000002e-06 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745539+0000", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032758879999999997 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745577+0000", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7704999999999999e-05 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745628+0000", 2024-09-12T09:30:00.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0850000000000003e-05 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10981 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9797632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9342262330000004, 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974687000000003, 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10981 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:00.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712788+0000", 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712792+0000", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5299999999999998e-06 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712796+0000", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9199999999999997e-06 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745657+0000", 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032860633 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745701+0000", 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4546999999999999e-05 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745760+0000", 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9228000000000003e-05 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10982 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9863168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9342090929999998, 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033077600999999998, 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10982 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.688 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1300000000000001e-07 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712805+0000", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712811+0000", 2024-09-12T09:30:00.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712815+0000", 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9720000000000003e-06 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745790+0000", 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974742000000001 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745828+0000", 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7870000000000002e-05 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745880+0000", 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2512999999999997e-05 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10983 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9928704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9341909259999994, 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033178996000000002, 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10983 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2899999999999999e-07 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712823+0000", 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.522e-06 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712827+0000", 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6e-06 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712831+0000", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8580000000000002e-06 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745910+0000", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033078308000000001 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745948+0000", 2024-09-12T09:30:00.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8012000000000001e-05 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746000+0000", 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2367000000000002e-05 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10984 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9994240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:00.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9341754299999998, 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033283132, 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10984 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712837+0000", 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2599999999999998e-07 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712838+0000", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.505e-06 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712843+0000", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712847+0000", 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-12T09:30:00.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746028+0000", 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033181575999999997 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746066+0000", 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7892000000000002e-05 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746120+0000", 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3375999999999999e-05 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10985 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10059776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9341599669999994, 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033383775999999997, 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10985 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2399999999999999e-07 2024-09-12T09:30:00.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712854+0000", 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712859+0000", 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712862+0000", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8130000000000002e-06 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746148+0000", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033285631000000003 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746185+0000", 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7381e-05 2024-09-12T09:30:00.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746236+0000", 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0334999999999998e-05 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10986 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10125312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9341444820000007, 2024-09-12T09:30:00.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033485506999999998, 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10986 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:00.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712868+0000", 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712869+0000", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.598e-06 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712876+0000", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.657e-06 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712883+0000", 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4050000000000003e-06 2024-09-12T09:30:00.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746264+0000", 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033380552000000001 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746301+0000", 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7187000000000002e-05 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746353+0000", 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1745e-05 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10987 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10190848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9341167670000008, 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033575365000000003, 2024-09-12T09:30:00.712 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10987 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712896+0000", 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4799999999999998e-07 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712899+0000", 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9409999999999999e-06 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712910+0000", 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712918+0000", 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8709999999999997e-06 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746381+0000", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033463443000000002 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746419+0000", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7415e-05 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746471+0000", 2024-09-12T09:30:00.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1903000000000001e-05 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10988 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10256384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9340840589999999, 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033660336999999999, 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10988 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712931+0000", 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6309999999999999e-06 2024-09-12T09:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712941+0000", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7149999999999993e-06 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712948+0000", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6129999999999998e-06 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746499+0000", 2024-09-12T09:30:00.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033550330000000003 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746536+0000", 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7413000000000002e-05 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746588+0000", 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2154999999999998e-05 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10989 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10321920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9340537629999996, 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033748256999999997, 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10989 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712959+0000", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9100000000000004e-07 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712967+0000", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1839999999999999e-06 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712988+0000", 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1356999999999999e-05 2024-09-12T09:30:00.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712997+0000", 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746617+0000", 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033619818000000003 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746654+0000", 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7419000000000002e-05 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746706+0000", 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2318000000000003e-05 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10990 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10387456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9340023409999993, 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033815079999999997, 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10990 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8899999999999999e-07 2024-09-12T09:30:00.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713015+0000", 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2950000000000004e-06 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713025+0000", 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713033+0000", 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746735+0000", 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033701704999999998 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746772+0000", 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7573000000000001e-05 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746825+0000", 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2268999999999997e-05 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10991 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10452992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9339681740000003, 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033896353999999997, 2024-09-12T09:30:00.731 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10991 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713549+0000", 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050486900000000002 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713554+0000", 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8660000000000002e-06 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713569+0000", 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4497e-05 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713579+0000", 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0074e-05 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746853+0000", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033273529000000003 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746889+0000", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746940+0000", 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0640999999999998e-05 2024-09-12T09:30:00.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10992 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10518528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9334152630000006, 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033468748999999999, 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10992 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713601+0000", 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713612+0000", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0429e-05 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713619+0000", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000001e-06 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746969+0000", 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033349733999999999 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747007+0000", 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7576000000000001e-05 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747065+0000", 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8770999999999997e-05 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10993 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10584064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9333813880000008, 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033553980999999997, 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10993 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720607+0000", 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069764909999999996 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720616+0000", 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2289999999999992e-06 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720631+0000", 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4483e-05 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720640+0000", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3749999999999992e-06 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747095+0000", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026454688000000001 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747133+0000", 2024-09-12T09:30:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7719000000000003e-05 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747185+0000", 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1996000000000002e-05 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:00.745 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:00.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:00.788 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:00.785+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:00.848 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:00.845+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:00.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:00.879 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:00.879 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:00.879 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:00.879 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:00.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:00.979 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:00.977+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:01.007 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:01.008 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:01.024 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:01.043 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:01.043 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:01.050 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:01.045+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:01.059 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:01.060 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:01.060 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:01.060 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:01.060 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:01.061 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:01.061 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:01.074 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:01.083 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:01.101 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:01.121 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:01.141 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:01.150 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:01.145+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:01.249 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:01.249 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:01.250 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:01.245+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:01.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:01.351 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:01.349+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:01.429 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:01.430 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:01.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:01.452 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:01.449+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:01.508 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-12T09:30:01.553 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:01.549+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:01.654 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:01.649+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:01.657 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:01.657 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:01.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:01.754 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:01.749+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:01.785 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:01.785 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:01.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:01.895 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:01.893+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:01.956 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:01.953+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:01.969 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:01.969 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:01.983 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:02.057 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:02.053+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:02.144 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.144 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.158 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:02.153+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:02.158 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:02.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:30:02.259 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:02.253+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:02.276 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.276 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:02.359 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:02.357+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:02.408 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.408 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:02.461 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:02.457+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:02.490 INFO:teuthology.orchestra.run.smithi007.stdout:870 read 0xb36174f thru 0xb363f98 (0x284a bytes) 2024-09-12T09:30:02.491 INFO:teuthology.orchestra.run.smithi007.stdout:872 punch from 0xbd53560 to 0xbd57ad7, (0x4577 bytes) 2024-09-12T09:30:02.496 INFO:teuthology.orchestra.run.smithi007.stdout:878 read 0x908ae2f thru 0x9095693 (0xa865 bytes) 2024-09-12T09:30:02.496 INFO:teuthology.orchestra.run.smithi007.stdout:880 write 0x9263c14 thru 0x9264f2c (0x1319 bytes) 2024-09-12T09:30:02.502 INFO:teuthology.orchestra.run.smithi007.stdout:881 writesame 0xbbe1389 thru 0xbbe2645 data_size 0x27(0x12bd bytes) 2024-09-12T09:30:02.515 INFO:teuthology.orchestra.run.smithi007.stdout:882 write 0xdd8d8da thru 0xdd95b0d (0x8234 bytes) 2024-09-12T09:30:02.522 INFO:teuthology.orchestra.run.smithi007.stdout:883 punch from 0xb7db663 to 0xb7e011d, (0x4aba bytes) 2024-09-12T09:30:02.522 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.522 INFO:teuthology.orchestra.run.smithi007.stdout:885 read 0xaecdb8e thru 0xaedc2d3 (0xe746 bytes) 2024-09-12T09:30:02.522 INFO:teuthology.orchestra.run.smithi007.stdout:886 write 0x876543a thru 0x8769553 (0x411a bytes) 2024-09-12T09:30:02.533 INFO:teuthology.orchestra.run.smithi007.stdout:888 writesame 0xc2875c0 thru 0xc2884e5 data_size 0x7(0xf26 bytes) 2024-09-12T09:30:02.542 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.542 INFO:teuthology.orchestra.run.smithi007.stdout:890 read 0x253155f thru 0x2534f8a (0x3a2c bytes) 2024-09-12T09:30:02.542 INFO:teuthology.orchestra.run.smithi007.stdout:891 read 0x6157bc3 thru 0x615b4a1 (0x38df bytes) 2024-09-12T09:30:02.543 INFO:teuthology.orchestra.run.smithi007.stdout:893 read 0x9d786c thru 0x9e1711 (0x9ea6 bytes) 2024-09-12T09:30:02.544 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.544 INFO:teuthology.orchestra.run.smithi007.stdout:895 write 0x27016c8 thru 0x270da9d (0xc3d6 bytes) 2024-09-12T09:30:02.552 INFO:teuthology.orchestra.run.smithi007.stdout:896 write 0xd17203d thru 0xd17aac3 (0x8a87 bytes) 2024-09-12T09:30:02.561 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.561 INFO:teuthology.orchestra.run.smithi007.stdout:899 read 0xd51ec6a thru 0xd5277cf (0x8b66 bytes) 2024-09-12T09:30:02.561 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.561 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:02.557+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:02.569 INFO:teuthology.orchestra.run.smithi007.stdout:902 write 0xdd9b439 thru 0xdda828b (0xce53 bytes) 2024-09-12T09:30:02.572 INFO:teuthology.orchestra.run.smithi007.stdout:903 read 0x8724f50 thru 0x87251ca (0x27b bytes) 2024-09-12T09:30:02.573 INFO:teuthology.orchestra.run.smithi007.stdout:907 read 0x465acfd thru 0x4660f2d (0x6231 bytes) 2024-09-12T09:30:02.574 INFO:teuthology.orchestra.run.smithi007.stdout:908 write 0x7baae02 thru 0x7bbac4b (0xfe4a bytes) 2024-09-12T09:30:02.583 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:02.583 INFO:teuthology.orchestra.run.smithi007.stdout:914 writesame 0x746fc1a thru 0x747a03b data_size 0x6(0xa422 bytes) 2024-09-12T09:30:02.597 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.597 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.603 INFO:teuthology.orchestra.run.smithi007.stdout:915 write 0xe2a8301 thru 0xe2acdc6 (0x4ac6 bytes) 2024-09-12T09:30:02.613 INFO:teuthology.orchestra.run.smithi007.stdout:917 writesame 0x75b2c34 thru 0x75b74bb data_size 0x2c(0x4888 bytes) 2024-09-12T09:30:02.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:02.619 INFO:teuthology.orchestra.run.smithi007.stdout:918 write 0x7a1efa3 thru 0x7a2ec2d (0xfc8b bytes) 2024-09-12T09:30:02.627 INFO:teuthology.orchestra.run.smithi007.stdout:919 clone 9 order 25 su 524288 sc 7 2024-09-12T09:30:02.652 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:30:02.663 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:02.661+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:02.743 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone8 from 0xe2acdc7 (overlap 0xdc54704) to 0x0 2024-09-12T09:30:02.743 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.743 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.764 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:02.761+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:02.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:02.865 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:02.861+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:02.925 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:02.926 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:02.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:02.966 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:02.965+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:03.067 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:03.061+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:03.111 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:03.111 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:03.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:03.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:03.168 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:03.165+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:03.269 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:03.265+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:03.289 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:03.289 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:03.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:03.318 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:03.335 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:03.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:03.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:03.369 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:03.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:03.370 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:03.365+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:03.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:03.389 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:03.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:03.410 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:03.431 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:03.434 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:03.434 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:03.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:03.452 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:03.470 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:03.465+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:03.567 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:03.567 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:03.571 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:03.569+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:03.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:03.672 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:03.669+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:03.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.66633912, 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:30:03.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:30:03.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:03.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.945787335999999, 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:03.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:30:03.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944283343999999, 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:30:03.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.733 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944260594999999, 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:30:03.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:30:03.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:03.738 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944237155, 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:30:03.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944213706999999, 2024-09-12T09:30:03.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:03.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:30:03.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944170271000001, 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:30:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:30:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.944133474000001, 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:30:03.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.876145233999999, 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:30:03.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.876089673999999, 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:30:03.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:30:03.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.876060424999999, 2024-09-12T09:30:03.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.876036720999998, 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:30:03.772 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:30:03.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.876016993, 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:30:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.875993471000001, 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:30:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.875970497000001, 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.875926308, 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:30:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:30:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:30:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.875891940999999, 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:30:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.822468571999998, 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:30:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:30:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.821903735999999, 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:30:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10623 3.4 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.195627537, 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017782925000000002, 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10623 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520647+0000", 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9369999999999998e-06 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520656+0000", 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520672+0000", 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6018000000000001e-05 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520680+0000", 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6079999999999998e-06 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520788+0000", 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010822 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520800+0000", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1753000000000001e-05 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.523800+0000", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030001049999999999 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.524016+0000", 2024-09-12T09:30:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021596300000000001 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527527+0000", 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003511424 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538252+0000", 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010724768000000001 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538311+0000", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9432999999999998e-05 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538380+0000", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8083000000000005e-05 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538428+0000", 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8906000000000002e-05 2024-09-12T09:30:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:03.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:03.820 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-12T09:30:03.820 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:03.817+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:03.920 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:03.917+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:03.954 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:03.954 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:03.954 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:03.954 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:03.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:04.021 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:04.017+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:04.122 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:04.117+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:04.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:04.147 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:04.147 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:04.147 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:04.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:04.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:04.188 INFO:teuthology.orchestra.run.smithi007.stdout:920 punch from 0x41f4feb to 0x41f536a, (0x37f bytes) 2024-09-12T09:30:04.193 INFO:teuthology.orchestra.run.smithi007.stdout:923 read 0x16bd348 thru 0x16be36f (0x1028 bytes) 2024-09-12T09:30:04.194 INFO:teuthology.orchestra.run.smithi007.stdout:924 read 0x3014759 thru 0x301bd32 (0x75da bytes) 2024-09-12T09:30:04.195 INFO:teuthology.orchestra.run.smithi007.stdout:926 write 0x6938ac7 thru 0x6941209 (0x8743 bytes) 2024-09-12T09:30:04.223 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:04.217+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:04.310 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1015 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'4, mlcod=27'2)", 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.885394954999999, 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001427214, 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422636+0000", 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422637+0000", 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.528e-06 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422646+0000", 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7080000000000003e-06 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422678+0000", 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1829000000000003e-05 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422692+0000", 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4116e-05 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422811+0000", 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011919899999999999 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.422852+0000", 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1016000000000002e-05 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.423982+0000", 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011297950000000001 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.424063+0000", 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1023e-05 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.842551538999999, 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:30:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:30:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1039 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'13, mlcod=28'11)", 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.822313186999999, 2024-09-12T09:30:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013168500000000001, 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485718+0000", 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485720+0000", 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1670000000000002e-06 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485733+0000", 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3183000000000001e-05 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485765+0000", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1866000000000003e-05 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485785+0000", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9830000000000002e-05 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485917+0000", 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013222599999999999 2024-09-12T09:30:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.485968+0000", 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1106000000000002e-05 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.486960+0000", 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099206899999999994 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.487034+0000", 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4402999999999999e-05 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.777887914000001, 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:30:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:30:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.773319032, 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:30:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.595657620000001, 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:30:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:30:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.572699255, 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:30:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.142350835999999, 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:30:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:3734 3.4 e33/23 3:21777318:::rbd_data.11912e4e65e9.0000000000000017:head v 33'1, mlcod=0'0)", 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.659142782, 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001459442, 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648890+0000", 2024-09-12T09:30:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.829e-06 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649057+0000", 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729099999999999 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649076+0000", 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8902e-05 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649083+0000", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.277e-06 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649153+0000", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9380000000000003e-05 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649177+0000", 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.425e-05 2024-09-12T09:30:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650298+0000", 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011215369999999999 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650347+0000", 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8976e-05 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:4220 3.4 e33/23 3:245d796b:::rbd_data.11912e4e65e9.0000000000000026:head v 33'2, mlcod=0'0)", 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.626237628999998, 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014342420000000001, 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681795+0000", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681925+0000", 2024-09-12T09:30:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012997400000000001 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681944+0000", 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9531e-05 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681954+0000", 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5149999999999995e-06 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682029+0000", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5421999999999997e-05 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682056+0000", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6942000000000001e-05 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683173+0000", 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001117093 2024-09-12T09:30:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683227+0000", 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000002e-05 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5962 3.4 e33/23 3:21146361:::rbd_data.11912e4e65e9.000000000000005d:head v 33'3, mlcod=33'1)", 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.524968131000001, 2024-09-12T09:30:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001433824, 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783064+0000", 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.798e-06 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783189+0000", 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012434600000000001 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783217+0000", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8218e-05 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783230+0000", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2965e-05 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783343+0000", 2024-09-12T09:30:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113309 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783366+0000", 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2625000000000001e-05 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784454+0000", 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001088356 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784496+0000", 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2206999999999998e-05 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5964 3.0 e33/23 3:02a4fae5:::rbd_data.11912e4e65e9.000000000000005f:head v 33'32, mlcod=31'29)", 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.517091281999999, 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001299197, 2024-09-12T09:30:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790939+0000", 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790941+0000", 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5510000000000001e-06 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791013+0000", 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2242000000000004e-05 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791043+0000", 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0028000000000001e-05 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791053+0000", 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0154e-05 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791195+0000", 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014127599999999999 2024-09-12T09:30:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.791216+0000", 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-05 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792186+0000", 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096983200000000001 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792239+0000", 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2814e-05 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.513577515000001, 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:30:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.512801883000002, 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:30:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:30:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.176885773, 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:30:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.170077096, 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.781129529999999, 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:30:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.777759497, 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:30:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:30:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.272139703000001, 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:30:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11694 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'17, mlcod=38'15)", 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.268852209, 2024-09-12T09:30:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015297240000000001, 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:04.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039276+0000", 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6244999999999998e-05 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039292+0000", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039302+0000", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9369999999999999e-06 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039368+0000", 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5711999999999997e-05 2024-09-12T09:30:04.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039387+0000", 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9386999999999999e-05 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040592+0000", 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012047270000000001 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040708+0000", 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116242 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.406 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:04.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:04.408 INFO:teuthology.orchestra.run.smithi007.stdout:927 writesame 0x1a2092b thru 0x1a2add9 data_size 0x45(0xa4af bytes) 2024-09-12T09:30:04.409 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:04.405+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:04.509 INFO:teuthology.orchestra.run.smithi007.stdout:928 punch from 0xb5eca10 to 0xb5efdba, (0x33aa bytes) 2024-09-12T09:30:04.509 INFO:teuthology.orchestra.run.smithi007.stdout:930 writesame 0xa4bb90a thru 0xa4cb47f data_size 0x3(0xfb76 bytes) 2024-09-12T09:30:04.509 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:04.505+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:04.527 INFO:teuthology.orchestra.run.smithi007.stdout:933 writesame 0x5f6c750 thru 0x5f70f99 data_size 0x13(0x484a bytes) 2024-09-12T09:30:04.588 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:04.588 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:04.588 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:04.588 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:04.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:04.633 INFO:teuthology.orchestra.run.smithi007.stdout:935 read 0xde556f8 thru 0xde59769 (0x4072 bytes) 2024-09-12T09:30:04.634 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:04.629+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:04.634 INFO:teuthology.orchestra.run.smithi007.stdout:936 writesame 0xa18be3c thru 0xa191994 data_size 0x5(0x5b59 bytes) 2024-09-12T09:30:04.641 INFO:teuthology.orchestra.run.smithi007.stdout:937 write 0x43a7d64 thru 0x43b5bd9 (0xde76 bytes) 2024-09-12T09:30:04.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:30:04.710 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:04.705+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:04.738 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:04.738 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:04.740 INFO:teuthology.orchestra.run.smithi007.stdout:938 punch from 0x9ab3527 to 0x9ac2091, (0xeb6a bytes) 2024-09-12T09:30:04.740 INFO:teuthology.orchestra.run.smithi007.stdout:939 read 0x72f89f7 thru 0x73067cd (0xddd7 bytes) 2024-09-12T09:30:04.742 INFO:teuthology.orchestra.run.smithi007.stdout:941 clone 10 order 23 su 32768 sc 3 2024-09-12T09:30:04.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:04.811 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:04.805+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:04.879 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10975 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9404416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164559997, 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032398173000000002, 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10975 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712637+0000", 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6899999999999997e-07 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712640+0000", 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712650+0000", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189e-05 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712657+0000", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3699999999999997e-06 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744949+0000", 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032291804 2024-09-12T09:30:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744986+0000", 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7211e-05 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745034+0000", 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7947e-05 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10976 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9469952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164511921000001, 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032464923999999999, 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10976 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712685+0000", 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712687+0000", 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8509999999999999e-06 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712695+0000", 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712699+0000", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5059999999999997e-06 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745063+0000", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032363839999999998 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745101+0000", 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7833999999999997e-05 2024-09-12T09:30:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745149+0000", 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8288999999999998e-05 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10977 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9535488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164490334, 2024-09-12T09:30:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032562971000000003, 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10977 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8500000000000002e-07 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712716+0000", 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5070000000000006e-06 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712721+0000", 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5079999999999996e-06 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712726+0000", 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8919999999999999e-06 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745178+0000", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032451498000000002 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745216+0000", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7973999999999999e-05 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745269+0000", 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3106999999999998e-05 2024-09-12T09:30:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10978 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9601024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164463913000001, 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032657185999999998, 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10978 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7099999999999997e-07 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712734+0000", 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5120000000000001e-06 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712741+0000", 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8040000000000004e-06 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712745+0000", 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3470000000000001e-06 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745298+0000", 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032552400000000002 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745336+0000", 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8191000000000001e-05 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745389+0000", 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3560999999999998e-05 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10979 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9666560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164444136, 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032757778000000001, 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10979 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712754+0000", 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5910000000000001e-06 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712759+0000", 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9960000000000001e-06 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712764+0000", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745418+0000", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032653765000000001 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745456+0000", 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8197e-05 2024-09-12T09:30:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745510+0000", 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3344999999999998e-05 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10980 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9732096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164426397, 2024-09-12T09:30:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032858152000000002, 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10980 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712772+0000", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712776+0000", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7079999999999996e-06 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712780+0000", 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0890000000000002e-06 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745539+0000", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032758879999999997 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745577+0000", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7704999999999999e-05 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745628+0000", 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0850000000000003e-05 2024-09-12T09:30:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10981 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9797632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164410301, 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974687000000003, 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10981 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-12T09:30:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712788+0000", 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712792+0000", 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5299999999999998e-06 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712796+0000", 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9199999999999997e-06 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745657+0000", 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032860633 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745701+0000", 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4546999999999999e-05 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745760+0000", 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9228000000000003e-05 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10982 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9863168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164393161, 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033077600999999998, 2024-09-12T09:30:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10982 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1300000000000001e-07 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712805+0000", 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712811+0000", 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712815+0000", 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9720000000000003e-06 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745790+0000", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974742000000001 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745828+0000", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7870000000000002e-05 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745880+0000", 2024-09-12T09:30:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2512999999999997e-05 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10983 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9928704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164374993999999, 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033178996000000002, 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10983 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2899999999999999e-07 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712823+0000", 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.522e-06 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712827+0000", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6e-06 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712831+0000", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8580000000000002e-06 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745910+0000", 2024-09-12T09:30:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033078308000000001 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745948+0000", 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8012000000000001e-05 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746000+0000", 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2367000000000002e-05 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10984 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9994240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164359498, 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033283132, 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10984 2024-09-12T09:30:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712837+0000", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2599999999999998e-07 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712838+0000", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.505e-06 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712843+0000", 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-12T09:30:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712847+0000", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746028+0000", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033181575999999997 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746066+0000", 2024-09-12T09:30:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7892000000000002e-05 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746120+0000", 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3375999999999999e-05 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10985 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10059776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164344034999999, 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033383775999999997, 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10985 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2399999999999999e-07 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712854+0000", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712859+0000", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712862+0000", 2024-09-12T09:30:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8130000000000002e-06 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746148+0000", 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033285631000000003 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746185+0000", 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7381e-05 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746236+0000", 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0334999999999998e-05 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10986 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10125312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.16432855, 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033485506999999998, 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10986 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712868+0000", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712869+0000", 2024-09-12T09:30:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.598e-06 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712876+0000", 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.657e-06 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712883+0000", 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4050000000000003e-06 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746264+0000", 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033380552000000001 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746301+0000", 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7187000000000002e-05 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746353+0000", 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1745e-05 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10987 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10190848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164300835000001, 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033575365000000003, 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10987 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712896+0000", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4799999999999998e-07 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712899+0000", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9409999999999999e-06 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712910+0000", 2024-09-12T09:30:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712918+0000", 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8709999999999997e-06 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746381+0000", 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033463443000000002 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746419+0000", 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7415e-05 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746471+0000", 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1903000000000001e-05 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10988 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10256384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164268127, 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033660336999999999, 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10988 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712931+0000", 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6309999999999999e-06 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712941+0000", 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7149999999999993e-06 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712948+0000", 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6129999999999998e-06 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746499+0000", 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033550330000000003 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746536+0000", 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7413000000000002e-05 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746588+0000", 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2154999999999998e-05 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10989 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10321920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164237830999999, 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033748256999999997, 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10989 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712959+0000", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9100000000000004e-07 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712967+0000", 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1839999999999999e-06 2024-09-12T09:30:04.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712988+0000", 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1356999999999999e-05 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712997+0000", 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746617+0000", 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033619818000000003 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746654+0000", 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7419000000000002e-05 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746706+0000", 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2318000000000003e-05 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10990 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10387456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164186408999999, 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033815079999999997, 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10990 2024-09-12T09:30:04.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8899999999999999e-07 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713015+0000", 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2950000000000004e-06 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713025+0000", 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713033+0000", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746735+0000", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033701704999999998 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746772+0000", 2024-09-12T09:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7573000000000001e-05 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746825+0000", 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2268999999999997e-05 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10991 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10452992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:04.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.164152242, 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033896353999999997, 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10991 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:04.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713549+0000", 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050486900000000002 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713554+0000", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8660000000000002e-06 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713569+0000", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4497e-05 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713579+0000", 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0074e-05 2024-09-12T09:30:04.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746853+0000", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033273529000000003 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746889+0000", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746940+0000", 2024-09-12T09:30:04.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0640999999999998e-05 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10992 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10518528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.163599331, 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033468748999999999, 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10992 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713601+0000", 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713612+0000", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0429e-05 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713619+0000", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000001e-06 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746969+0000", 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033349733999999999 2024-09-12T09:30:04.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747007+0000", 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7576000000000001e-05 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747065+0000", 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8770999999999997e-05 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10993 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10584064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.163565456000001, 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033553980999999997, 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10993 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720607+0000", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069764909999999996 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720616+0000", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2289999999999992e-06 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720631+0000", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4483e-05 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720640+0000", 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3749999999999992e-06 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747095+0000", 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026454688000000001 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747133+0000", 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7719000000000003e-05 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747185+0000", 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1996000000000002e-05 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:49995 3.15 e50/23 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head v 50'152, mlcod=49'150)", 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.29787333999999999, 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053471088999999999, 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:04.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611150+0000", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031827393000000002 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611191+0000", 2024-09-12T09:30:04.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0123000000000002e-05 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611215+0000", 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4664999999999998e-05 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611349+0000", 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013327799999999999 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611373+0000", 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4176999999999999e-05 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.632751+0000", 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021378476 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.632794+0000", 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2546999999999998e-05 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:04.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:04.974 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:04.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:04.976 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:04.973+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:05.077 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:05.073+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:05.105 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:05.106 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:05.106 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:05.106 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:05.119 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:05.177 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:05.177+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:05.195 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 7, 0] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:05.195 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:05.195 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [1, 2, 3, 4, 5, 7, 0] 2024-09-12T09:30:05.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-12T09:30:05.278 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:05.273+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:05.288 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:05.288 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:05.289 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:05.289 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:05.289 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:05.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:05.387 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:05.385+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:05.470 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2759 3.3 3:c032df1e:::rbd_data.1191d634a7ff.0000000000000000:5 [read 933888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578368+0000", 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890596879, 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031992347999999997, 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2759 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578368+0000", 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578368+0000", 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578369+0000", 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7400000000000003e-07 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578372+0000", 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1379999999999999e-06 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578385+0000", 2024-09-12T09:30:05.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3342e-05 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578398+0000", 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2945999999999999e-05 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610008+0000", 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031609402000000002 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610173+0000", 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165095 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610361+0000", 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000187751 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2760 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578431+0000", 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890533605000002, 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031990686999999997, 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2760 2024-09-12T09:30:05.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578431+0000", 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578431+0000", 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578433+0000", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.449e-06 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578437+0000", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.442e-06 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578453+0000", 2024-09-12T09:30:05.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5651999999999999e-05 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578464+0000", 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1231e-05 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610411+0000", 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031947235999999997 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610422+0000", 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0677e-05 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2761 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578513+0000", 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890451568, 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031977477999999997, 2024-09-12T09:30:05.483 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2761 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578513+0000", 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578513+0000", 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578515+0000", 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1850000000000001e-06 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578521+0000", 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7769999999999997e-06 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578538+0000", 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6109000000000001e-05 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578547+0000", 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7680000000000003e-06 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610473+0000", 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031925991000000001 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610491+0000", 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7648000000000001e-05 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2762 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.487 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578588+0000", 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890377329, 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031988721999999997, 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2762 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578588+0000", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578588+0000", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578589+0000", 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.26e-06 2024-09-12T09:30:05.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578594+0000", 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2399999999999998e-06 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578610+0000", 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5469e-05 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578621+0000", 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1588e-05 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610561+0000", 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031940061999999998 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610576+0000", 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5102999999999999e-05 2024-09-12T09:30:05.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2763 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578682+0000", 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890282927000001, 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031945868000000002, 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2763 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578682+0000", 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578682+0000", 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578683+0000", 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578687+0000", 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0799999999999999e-06 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578701+0000", 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3997e-05 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578708+0000", 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610618+0000", 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031910663999999998 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610628+0000", 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5580000000000006e-06 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2764 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578718+0000", 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890246641000001, 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031966625999999998, 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.496 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2764 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578718+0000", 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578718+0000", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578719+0000", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3099999999999997e-07 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578723+0000", 2024-09-12T09:30:05.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.366e-06 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578738+0000", 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5311e-05 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578744+0000", 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.64e-06 2024-09-12T09:30:05.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610672+0000", 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031927658999999997 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610685+0000", 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2918999999999999e-05 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2765 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578758+0000", 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890207377999999, 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031982396000000003, 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2765 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578758+0000", 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578758+0000", 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578759+0000", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4339999999999999e-06 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578763+0000", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2649999999999998e-06 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578778+0000", 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4683e-05 2024-09-12T09:30:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578791+0000", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.342e-05 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610726+0000", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031934214000000002 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610740+0000", 2024-09-12T09:30:05.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.438e-05 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2766 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578807+0000", 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890158433, 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032000237000000001, 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.505 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2766 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578807+0000", 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578807+0000", 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578808+0000", 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578812+0000", 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0799999999999999e-06 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578826+0000", 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.45e-05 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578837+0000", 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0762e-05 2024-09-12T09:30:05.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610796+0000", 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031958746000000003 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610807+0000", 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1243000000000001e-05 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2767 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578899+0000", 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890065831000001, 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031980244999999997, 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.510 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2767 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578899+0000", 2024-09-12T09:30:05.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578899+0000", 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578900+0000", 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578905+0000", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9089999999999999e-06 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578915+0000", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8570000000000007e-06 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578922+0000", 2024-09-12T09:30:05.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0140000000000001e-06 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610860+0000", 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031938659000000001 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610879+0000", 2024-09-12T09:30:05.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9032e-05 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2768 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.578932+0000", 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.890033277000001, 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032002192999999998, 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2768 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578932+0000", 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578932+0000", 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578932+0000", 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578935+0000", 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.357e-06 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578945+0000", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7710000000000001e-06 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.578957+0000", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2177e-05 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610925+0000", 2024-09-12T09:30:05.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031968208999999997 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610934+0000", 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2769 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579020+0000", 2024-09-12T09:30:05.519 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889945389000001, 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031983777999999997, 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2769 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579020+0000", 2024-09-12T09:30:05.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579020+0000", 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579021+0000", 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5400000000000001e-06 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579034+0000", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3125999999999999e-05 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579054+0000", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0074000000000001e-05 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579067+0000", 2024-09-12T09:30:05.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2349e-05 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.610984+0000", 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031916858999999999 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611003+0000", 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9830000000000002e-05 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2770 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579086+0000", 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889879246, 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031946928999999999, 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2770 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579086+0000", 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579086+0000", 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579087+0000", 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.097e-06 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579092+0000", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9400000000000001e-06 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579106+0000", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4248e-05 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579113+0000", 2024-09-12T09:30:05.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.737e-06 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611022+0000", 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031909449999999999 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611033+0000", 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0457e-05 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2771 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579137+0000", 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889827791999998, 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031966613999999997, 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2771 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579137+0000", 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579137+0000", 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579138+0000", 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579141+0000", 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8219999999999998e-06 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579155+0000", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3740000000000001e-05 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579161+0000", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8950000000000001e-06 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611087+0000", 2024-09-12T09:30:05.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031925228 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611104+0000", 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7186000000000001e-05 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2772 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579171+0000", 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889793877999999, 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031965782999999998, 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2772 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579171+0000", 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579171+0000", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579172+0000", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579174+0000", 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5919999999999999e-06 2024-09-12T09:30:05.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579195+0000", 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1033999999999999e-05 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579206+0000", 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0876999999999999e-05 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611125+0000", 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031918982999999998 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611137+0000", 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1666e-05 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2773 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579262+0000", 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889702626999998, 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031923751, 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2773 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579262+0000", 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579262+0000", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579264+0000", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3510000000000001e-06 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579268+0000", 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4830000000000001e-06 2024-09-12T09:30:05.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579284+0000", 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5823000000000001e-05 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579296+0000", 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1868000000000001e-05 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611166+0000", 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031870278000000002 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611186+0000", 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9947999999999999e-05 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2774 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579377+0000", 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889587984999999, 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031843220999999998, 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2774 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579377+0000", 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579377+0000", 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579378+0000", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4300000000000002e-07 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579381+0000", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.937e-06 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579395+0000", 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4080000000000001e-05 2024-09-12T09:30:05.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579402+0000", 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611210+0000", 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031808101999999998 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611220+0000", 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0274e-05 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2775 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579413+0000", 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889552025, 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031863882000000003, 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2775 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579413+0000", 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579413+0000", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579414+0000", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5099999999999999e-07 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579416+0000", 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-09-12T09:30:05.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579436+0000", 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0231999999999999e-05 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579453+0000", 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6756999999999999e-05 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611256+0000", 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031802658999999997 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611277+0000", 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1112e-05 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2776 3.3 3.78fb4c03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:05.552 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.579496+0000", 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.889468932, 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0318248, 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2776 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579496+0000", 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579496+0000", 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579498+0000", 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.432e-06 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579502+0000", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6140000000000002e-06 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579518+0000", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5651e-05 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.579531+0000", 2024-09-12T09:30:05.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3386e-05 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611314+0000", 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031782530000000003 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.611321+0000", 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.187e-06 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.179913255, 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:05.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:05.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:50126 3.e e50/23 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head v 50'106, mlcod=49'104)", 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.684981+0000", 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.78398417300000001, 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.056428853000000001, 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.684981+0000", 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.684981+0000", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.684984+0000", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4180000000000001e-06 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.707963+0000", 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022978826000000001 2024-09-12T09:30:05.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.708003+0000", 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9651999999999999e-05 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.708022+0000", 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9697000000000001e-05 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.708238+0000", 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215214 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.708277+0000", 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9774000000000002e-05 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741219+0000", 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032941612000000002 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741410+0000", 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019065999999999999 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.566 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:05.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:05.568 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:30:05.517+0000 7f3454e55640 -1 mon.a@0(leader).osd e50 definitely_dead 0 2024-09-12T09:30:05.568 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:05.565+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:05.569 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:30:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.7. 2024-09-12T09:30:05.668 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:05.665+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:05.720 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:05.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:05.744 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:05.744 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:05.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50188 3.d 3.262efded (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-12T09:30:05.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:05.706159+0000", 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039241019000000002, 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039263438999999997, 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50188 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:05.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706159+0000", 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706159+0000", 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:05.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706161+0000", 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.282e-06 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706167+0000", 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0929999999999999e-06 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706182+0000", 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4846e-05 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706189+0000", 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2989999999999999e-06 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706257+0000", 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7935999999999995e-05 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:05.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:05.706267+0000", 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0008e-05 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:30:05.749 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:05.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:05.752 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:05.759 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:05.769 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:05.787 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:05.787 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:05.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:05.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:05.807 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:05.805+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:05.825 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:05.843 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:05.908 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:05.905+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:05.927 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:05.927 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:05.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:05.987 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:05.985+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:06.065 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.066 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.071 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:06.065+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:06.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:06.172 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:06.169+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:06.245 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.245 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:06.303 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:06.301+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:06.374 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:06.369+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:06.429 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.429 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.443 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:06.488 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:06.485+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:06.576 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:06.573+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:06.611 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.611 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.626 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:06.683 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:06.673+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:06.753 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone9 from 0xe2acdc7 (overlap 0xe2acdc7) to 0x5aab8b6 2024-09-12T09:30:06.762 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.762 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.778 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:06.779 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:06.773+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:06.879 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:06.873+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:06.905 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:06.905 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:06.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:06.980 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:06.977+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:07.040 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.041 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:07.100 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:07.097+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:07.182 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:07.181+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:07.190 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.190 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:07.282 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:07.281+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:07.376 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.376 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.384 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:07.381+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:07.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:07.484 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:07.481+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:07.518 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.518 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:07.620 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:07.617+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:07.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:30:07.686 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:07.681+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:07.724 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.724 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:07.801 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:07.797+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:07.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:07.886 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:07.881+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:07.969 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:07.987 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:07.981+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:07.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:08.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:08.088 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:08.085+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:08.089 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:08.089 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:08.109 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:08.141 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-12T09:30:08.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:08.144 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:08.144 INFO:teuthology.orchestra.run.smithi007.stdout:942 punch from 0x537c035 to 0x537d00a, (0xfd5 bytes) 2024-09-12T09:30:08.149 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:08.149 INFO:teuthology.orchestra.run.smithi007.stdout:944 punch from 0x1819911 to 0x181e418, (0x4b07 bytes) 2024-09-12T09:30:08.149 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:08.149 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:08.150 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:08.150 INFO:teuthology.orchestra.run.smithi007.stdout:948 writesame 0xaaecb49 thru 0xaaef55d data_size 0x9(0x2a15 bytes) 2024-09-12T09:30:08.175 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:08.187 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:820 3.0 3:023b8689:::rbd_data.1191192d73a0.0000000000000024:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.137009906999999, 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017048896000000001, 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 820 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049934+0000", 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049936+0000", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049946+0000", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049966+0000", 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9385000000000002e-05 2024-09-12T09:30:08.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.049976+0000", 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050043+0000", 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7601999999999994e-05 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.050117+0000", 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3665999999999996e-05 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.052795+0000", 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026778280000000002 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066818+0000", 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014023169 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066874+0000", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6434000000000002e-05 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066918+0000", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3849e-05 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.066983+0000", 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4753999999999997e-05 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5837 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2310144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.416458123000002, 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018263162999999999, 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5837 2024-09-12T09:30:08.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770486+0000", 2024-09-12T09:30:08.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770488+0000", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1069999999999999e-06 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770495+0000", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.770501+0000", 2024-09-12T09:30:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.066e-06 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.777992+0000", 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074913449999999999 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.778076+0000", 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3570000000000001e-05 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788749+0000", 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010673243000000001 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5889 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 2973696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414954131000002, 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020706485, 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5889 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771990+0000", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771992+0000", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1629999999999999e-06 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.771999+0000", 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.579e-06 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772004+0000", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0669999999999999e-06 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788396+0000", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016392423 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788451+0000", 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4314999999999998e-05 2024-09-12T09:30:08.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.792696+0000", 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042454609999999999 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5890 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3006464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414931381999999, 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086603999999998, 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5890 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772012+0000", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772013+0000", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772016+0000", 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772023+0000", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772028+0000", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788885+0000", 2024-09-12T09:30:08.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016857365999999999 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.788959+0000", 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4189000000000004e-05 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789099+0000", 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000139946 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5891 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3039232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414907941999999, 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017293842, 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5891 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772036+0000", 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-12T09:30:08.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772039+0000", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.61e-06 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772046+0000", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1119999999999998e-06 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772051+0000", 2024-09-12T09:30:08.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0069999999999997e-06 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789126+0000", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017074580999999998 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789197+0000", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1265000000000003e-05 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789330+0000", 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013281 2024-09-12T09:30:08.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5892 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3072000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414884493999999, 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017522388999999999, 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5892 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772059+0000", 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772060+0000", 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772062+0000", 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6240000000000002e-06 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772069+0000", 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0849999999999999e-06 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772075+0000", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789356+0000", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017281210000000002 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789435+0000", 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9348e-05 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789582+0000", 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014648999999999999 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5893 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3104768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414841058, 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017718811000000001, 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5893 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772103+0000", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772107+0000", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2260000000000001e-06 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772118+0000", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1165999999999999e-05 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772126+0000", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789609+0000", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017482793 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789681+0000", 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2663000000000005e-05 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789822+0000", 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140085 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:5894 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3137536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.414804261, 2024-09-12T09:30:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017930952, 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5894 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772140+0000", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000003e-07 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772144+0000", 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5980000000000001e-06 2024-09-12T09:30:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772151+0000", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3540000000000002e-06 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.772157+0000", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789849+0000", 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017691993 2024-09-12T09:30:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.789925+0000", 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6493000000000001e-05 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.790071+0000", 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014516999999999999 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7177 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346816020999999, 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017582199, 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7177 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840128+0000", 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840129+0000", 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7000000000000004e-07 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840157+0000", 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8447999999999999e-05 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840167+0000", 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6180000000000001e-06 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840173+0000", 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3790000000000003e-06 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856457+0000", 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016283717999999999 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856705+0000", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024806699999999998 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857710+0000", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001005199 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7178 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346760460999999, 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016814499, 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7178 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840183+0000", 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840184+0000", 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840186+0000", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840193+0000", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4109999999999999e-06 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840202+0000", 2024-09-12T09:30:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6840000000000002e-06 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856845+0000", 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016642693 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856923+0000", 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859000000000001e-05 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.856998+0000", 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5154000000000006e-05 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7179 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346731212000002, 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016947145, 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7179 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840213+0000", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840215+0000", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.012e-06 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840222+0000", 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3239999999999997e-06 2024-09-12T09:30:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840228+0000", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857024+0000", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016796028000000001 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857096+0000", 2024-09-12T09:30:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.271e-05 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857160+0000", 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3430999999999996e-05 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7180 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346707508000001, 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017008533999999999, 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7180 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840236+0000", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840237+0000", 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840239+0000", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840244+0000", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4309999999999996e-06 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840248+0000", 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9380000000000002e-06 2024-09-12T09:30:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857175+0000", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01692724 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857213+0000", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8112999999999999e-05 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857245+0000", 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1989999999999997e-05 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7181 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4055040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.34668778, 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017086816000000001, 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7181 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840256+0000", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840259+0000", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.289e-06 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840266+0000", 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840271+0000", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0610000000000003e-06 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857258+0000", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016986765000000001 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857307+0000", 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9398999999999999e-05 2024-09-12T09:30:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857343+0000", 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5812999999999997e-05 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7182 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346664258000001, 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017142887999999998, 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7182 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840280+0000", 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840282+0000", 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3269999999999999e-06 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840287+0000", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7770000000000002e-06 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840292+0000", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857356+0000", 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01706388 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857392+0000", 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100999999999999e-05 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857422+0000", 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0049e-05 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7183 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346641284, 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017196705999999999, 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7183 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840303+0000", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840305+0000", 2024-09-12T09:30:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840318+0000", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3033e-05 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840336+0000", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7944e-05 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857435+0000", 2024-09-12T09:30:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017099223 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857470+0000", 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4303e-05 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857499+0000", 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9706999999999999e-05 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7184 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4153344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346597095, 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017229385999999999, 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7184 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840347+0000", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840356+0000", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2660000000000001e-06 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840363+0000", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840371+0000", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6809999999999998e-06 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857512+0000", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017140913000000001 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857546+0000", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4294000000000001e-05 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857576+0000", 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9938999999999999e-05 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:7185 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.346562727999999, 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017270259, 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7185 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840381+0000", 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840382+0000", 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5199999999999996e-07 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840384+0000", 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3719999999999999e-06 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840391+0000", 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9179999999999997e-06 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.840396+0000", 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3550000000000004e-06 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857588+0000", 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017192033999999998 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857622+0000", 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3441000000000001e-05 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.857651+0000", 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9487000000000001e-05 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8438 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 4349952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.293139359000001, 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017344884000000001, 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8438 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893804+0000", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893805+0000", 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:30:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893807+0000", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.03e-06 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893814+0000", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9140000000000002e-06 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.893820+0000", 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6879999999999996e-06 2024-09-12T09:30:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906525+0000", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012705064 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.906601+0000", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6503999999999998e-05 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911149+0000", 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045482359999999998 2024-09-12T09:30:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:8465 3.0 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:5 [read 5234688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.292574522999999, 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016733959999999999, 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 8465 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894369+0000", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894370+0000", 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894375+0000", 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1640000000000003e-06 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.894377+0000", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4210000000000002e-06 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911014+0000", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016637267000000001 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911054+0000", 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9218000000000003e-05 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.911103+0000", 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9753999999999998e-05 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10623 3.4 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.666298324, 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017782925000000002, 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10623 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520645+0000", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520647+0000", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9369999999999998e-06 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520656+0000", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7069999999999998e-06 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520672+0000", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6018000000000001e-05 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520680+0000", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6079999999999998e-06 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520788+0000", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010822 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.520800+0000", 2024-09-12T09:30:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1753000000000001e-05 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.523800+0000", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030001049999999999 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.524016+0000", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021596300000000001 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527527+0000", 2024-09-12T09:30:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003511424 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538252+0000", 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010724768000000001 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538311+0000", 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9432999999999998e-05 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538380+0000", 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8083000000000005e-05 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.538428+0000", 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8906000000000002e-05 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.278 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:08.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:08.279 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:08.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:08.280 INFO:teuthology.orchestra.run.smithi007.stdout:950 write 0xa63b104 thru 0xa64b01a (0xff17 bytes) 2024-09-12T09:30:08.281 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:08.281 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:08.282 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:08.277+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:08.331 INFO:teuthology.orchestra.run.smithi007.stdout:951 read 0x98728a2 thru 0x987c77c (0x9edb bytes) 2024-09-12T09:30:08.342 INFO:teuthology.orchestra.run.smithi007.stdout:952 write 0xee5a6f5 thru 0xee670b8 (0xc9c4 bytes) 2024-09-12T09:30:08.375 INFO:teuthology.orchestra.run.smithi007.stdout:957 trunc from 0xee670b9 to 0x943ee29 2024-09-12T09:30:08.380 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:08.377+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:08.393 INFO:teuthology.orchestra.run.smithi007.stdout:958 read 0x23f0da5 thru 0x23f5957 (0x4bb3 bytes) 2024-09-12T09:30:08.394 INFO:teuthology.orchestra.run.smithi007.stdout:959 punch from 0x635873 to 0x63ff22, (0xa6af bytes) 2024-09-12T09:30:08.394 INFO:teuthology.orchestra.run.smithi007.stdout:961 read 0x66380e5 thru 0x66402cd (0x81e9 bytes) 2024-09-12T09:30:08.394 INFO:teuthology.orchestra.run.smithi007.stdout:962 read 0x51d42d2 thru 0x51dff06 (0xbc35 bytes) 2024-09-12T09:30:08.396 INFO:teuthology.orchestra.run.smithi007.stdout:966 trunc from 0x943ee29 to 0x95635d5 2024-09-12T09:30:08.402 INFO:teuthology.orchestra.run.smithi007.stdout:967 write 0x8f63af0 thru 0x8f6e5e8 (0xaaf9 bytes) 2024-09-12T09:30:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:968 read 0x3608a9e thru 0x361274f (0x9cb2 bytes) 2024-09-12T09:30:08.473 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:08.473 INFO:teuthology.orchestra.run.smithi007.stdout:972 clone 11 order 23 su 2097152 sc 9 2024-09-12T09:30:08.474 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:08.474 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:08.482 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:08.477+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:08.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:08.583 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:08.581+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:08.639 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:08.640 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:08.640 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:08.640 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:08.640 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:08.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:30:08.683 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:08.681+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:08.775 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:08.775 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:08.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.309136834, 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:30:08.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.244473208999999, 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:30:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.239904327000001, 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:30:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.062242914999999, 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:30:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:30:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.039284550000001, 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:30:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.608936131, 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:30:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:3734 3.4 e33/23 3:21777318:::rbd_data.11912e4e65e9.0000000000000017:head v 33'1, mlcod=0'0)", 2024-09-12T09:30:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.125728077000002, 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001459442, 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648888+0000", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.648890+0000", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.829e-06 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649057+0000", 2024-09-12T09:30:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729099999999999 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649076+0000", 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8902e-05 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649083+0000", 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.277e-06 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649153+0000", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9380000000000003e-05 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.649177+0000", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.425e-05 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650298+0000", 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011215369999999999 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.650347+0000", 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8976e-05 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:4220 3.4 e33/23 3:245d796b:::rbd_data.11912e4e65e9.0000000000000026:head v 33'2, mlcod=0'0)", 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.092822924, 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014342420000000001, 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681793+0000", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681795+0000", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.655e-06 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681925+0000", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012997400000000001 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681944+0000", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9531e-05 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.681954+0000", 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5149999999999995e-06 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682029+0000", 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5421999999999997e-05 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.682056+0000", 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6942000000000001e-05 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683173+0000", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001117093 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.683227+0000", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000002e-05 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:5962 3.4 e33/23 3:21146361:::rbd_data.11912e4e65e9.000000000000005d:head v 33'3, mlcod=33'1)", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.991553425999999, 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001433824, 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783063+0000", 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783064+0000", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.798e-06 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783189+0000", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012434600000000001 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783217+0000", 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8218e-05 2024-09-12T09:30:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783230+0000", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2965e-05 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783343+0000", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000113309 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.783366+0000", 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2625000000000001e-05 2024-09-12T09:30:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784454+0000", 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001088356 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.784496+0000", 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2206999999999998e-05 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.980162809999999, 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:30:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.979387178, 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:30:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.643471068, 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:30:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.636662391000002, 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:30:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.247714824999999, 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:30:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.244344792, 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:30:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.738724997999999, 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:30:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:30:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11694 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'17, mlcod=38'15)", 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.735437504, 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015297240000000001, 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039276+0000", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6244999999999998e-05 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039292+0000", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039302+0000", 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9369999999999999e-06 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039368+0000", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5711999999999997e-05 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039387+0000", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9386999999999999e-05 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040592+0000", 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012047270000000001 2024-09-12T09:30:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040708+0000", 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116242 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.14 53/52 [PushOp(3:2cab5d4e:::rbd_data.11914330b43e.0000000000000013:9, version: 45'206, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2cab5d4e:::rbd_data.11914330b43e.0000000000000013:9@45'206, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.65463162600000002, 2024-09-12T09:30:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001534037, 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119985+0000", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5099999999999999e-07 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120704+0000", 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071853100000000001 2024-09-12T09:30:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120730+0000", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6276999999999999e-05 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120756+0000", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5981000000000001e-05 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120840+0000", 2024-09-12T09:30:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3794000000000004e-05 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120849+0000", 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5189999999999998e-06 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.121518+0000", 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066918399999999997 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.47520205599999998, 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:30:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51963 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'64, mlcod=49'62)", 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.38955554999999997, 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001823409, 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385062+0000", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3400000000000001e-06 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385070+0000", 2024-09-12T09:30:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7789999999999996e-06 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385089+0000", 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9561000000000001e-05 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385099+0000", 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385166+0000", 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385199+0000", 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2799999999999998e-05 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.386852+0000", 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016533749999999999 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.386884+0000", 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609000000000002e-05 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:08.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:08.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:08.867 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:08.861+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:08.968 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:08.965+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:09.035 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:09.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:09.068 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:09.069+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:09.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:09.161 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:09.169 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:09.165+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:09.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:09.270 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:09.265+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:09.342 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:09.342 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:09.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:09.342 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10975 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9404416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627567454000001, 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032398173000000002, 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10975 2024-09-12T09:30:09.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712636+0000", 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712637+0000", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6899999999999997e-07 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712640+0000", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712650+0000", 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189e-05 2024-09-12T09:30:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712657+0000", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3699999999999997e-06 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744949+0000", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032291804 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.744986+0000", 2024-09-12T09:30:09.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7211e-05 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745034+0000", 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7947e-05 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10976 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9469952~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:09.347 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627519377999999, 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032464923999999999, 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10976 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:09.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712684+0000", 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712685+0000", 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712687+0000", 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8509999999999999e-06 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712695+0000", 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712699+0000", 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5059999999999997e-06 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745063+0000", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032363839999999998 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745101+0000", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7833999999999997e-05 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745149+0000", 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8288999999999998e-05 2024-09-12T09:30:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10977 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9535488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627497791, 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032562971000000003, 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10977 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712706+0000", 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8500000000000002e-07 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712716+0000", 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5070000000000006e-06 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712721+0000", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5079999999999996e-06 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712726+0000", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8919999999999999e-06 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745178+0000", 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032451498000000002 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745216+0000", 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7973999999999999e-05 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745269+0000", 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3106999999999998e-05 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10978 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9601024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627471369999999, 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032657185999999998, 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10978 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712732+0000", 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7099999999999997e-07 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712734+0000", 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5120000000000001e-06 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712741+0000", 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8040000000000004e-06 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712745+0000", 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3470000000000001e-06 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745298+0000", 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032552400000000002 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745336+0000", 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8191000000000001e-05 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745389+0000", 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3560999999999998e-05 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10979 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9666560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:09.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627451593, 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032757778000000001, 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10979 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:09.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712752+0000", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712754+0000", 2024-09-12T09:30:09.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5910000000000001e-06 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712759+0000", 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9960000000000001e-06 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712764+0000", 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745418+0000", 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032653765000000001 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745456+0000", 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8197e-05 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745510+0000", 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3344999999999998e-05 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10980 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9732096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627433854, 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032858152000000002, 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10980 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712770+0000", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712772+0000", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712776+0000", 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7079999999999996e-06 2024-09-12T09:30:09.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712780+0000", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0890000000000002e-06 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745539+0000", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032758879999999997 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745577+0000", 2024-09-12T09:30:09.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7704999999999999e-05 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745628+0000", 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0850000000000003e-05 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10981 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9797632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627417758, 2024-09-12T09:30:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974687000000003, 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10981 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712786+0000", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712788+0000", 2024-09-12T09:30:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712792+0000", 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5299999999999998e-06 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712796+0000", 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9199999999999997e-06 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745657+0000", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032860633 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745701+0000", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4546999999999999e-05 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745760+0000", 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9228000000000003e-05 2024-09-12T09:30:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10982 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9863168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627400617999999, 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033077600999999998, 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10982 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712803+0000", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1300000000000001e-07 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712805+0000", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712811+0000", 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:09.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712815+0000", 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9720000000000003e-06 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745790+0000", 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032974742000000001 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745828+0000", 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7870000000000002e-05 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745880+0000", 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2512999999999997e-05 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10983 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9928704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.379 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627382450999999, 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033178996000000002, 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10983 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712821+0000", 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2899999999999999e-07 2024-09-12T09:30:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712823+0000", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.522e-06 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712827+0000", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6e-06 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712831+0000", 2024-09-12T09:30:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8580000000000002e-06 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745910+0000", 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033078308000000001 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.745948+0000", 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8012000000000001e-05 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746000+0000", 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2367000000000002e-05 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10984 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 9994240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627366954999999, 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033283132, 2024-09-12T09:30:09.384 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10984 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:09.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712836+0000", 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712837+0000", 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2599999999999998e-07 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712838+0000", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.505e-06 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712843+0000", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712847+0000", 2024-09-12T09:30:09.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746028+0000", 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033181575999999997 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746066+0000", 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7892000000000002e-05 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746120+0000", 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3375999999999999e-05 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10985 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10059776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627351491999999, 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033383775999999997, 2024-09-12T09:30:09.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10985 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712852+0000", 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2399999999999999e-07 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712854+0000", 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5209999999999999e-06 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712859+0000", 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712862+0000", 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8130000000000002e-06 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746148+0000", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033285631000000003 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746185+0000", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7381e-05 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746236+0000", 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0334999999999998e-05 2024-09-12T09:30:09.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10986 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10125312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627336007, 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033485506999999998, 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10986 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712867+0000", 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712868+0000", 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712869+0000", 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.598e-06 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712876+0000", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.657e-06 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712883+0000", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4050000000000003e-06 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746264+0000", 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033380552000000001 2024-09-12T09:30:09.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746301+0000", 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7187000000000002e-05 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746353+0000", 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1745e-05 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10987 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10190848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627308291999999, 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033575365000000003, 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10987 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712895+0000", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712896+0000", 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4799999999999998e-07 2024-09-12T09:30:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712899+0000", 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9409999999999999e-06 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712910+0000", 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1243999999999999e-05 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712918+0000", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8709999999999997e-06 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746381+0000", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033463443000000002 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746419+0000", 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7415e-05 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746471+0000", 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1903000000000001e-05 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10988 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10256384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627275583999999, 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033660336999999999, 2024-09-12T09:30:09.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10988 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:09.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712928+0000", 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712931+0000", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6309999999999999e-06 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712941+0000", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7149999999999993e-06 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712948+0000", 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6129999999999998e-06 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746499+0000", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033550330000000003 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746536+0000", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7413000000000002e-05 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746588+0000", 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2154999999999998e-05 2024-09-12T09:30:09.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10989 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10321920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627245288000001, 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033748256999999997, 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.408 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10989 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712958+0000", 2024-09-12T09:30:09.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712959+0000", 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9100000000000004e-07 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712967+0000", 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1839999999999999e-06 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712988+0000", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1356999999999999e-05 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.712997+0000", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5699999999999993e-06 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746617+0000", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033619818000000003 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746654+0000", 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7419000000000002e-05 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746706+0000", 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2318000000000003e-05 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10990 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10387456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:09.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627193865999999, 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033815079999999997, 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10990 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:09.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713010+0000", 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8899999999999999e-07 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713015+0000", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2950000000000004e-06 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713025+0000", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713033+0000", 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:30:09.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746735+0000", 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033701704999999998 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746772+0000", 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7573000000000001e-05 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746825+0000", 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2268999999999997e-05 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10991 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10452992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.627159699, 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033896353999999997, 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10991 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713044+0000", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713549+0000", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050486900000000002 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713554+0000", 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8660000000000002e-06 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713569+0000", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4497e-05 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713579+0000", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0074e-05 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746853+0000", 2024-09-12T09:30:09.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033273529000000003 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746889+0000", 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6878000000000002e-05 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746940+0000", 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0640999999999998e-05 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10992 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10518528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.626606788, 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033468748999999999, 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10992 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713597+0000", 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4000000000000001e-07 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713601+0000", 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713612+0000", 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0429e-05 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713619+0000", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7700000000000001e-06 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.746969+0000", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033349733999999999 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747007+0000", 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7576000000000001e-05 2024-09-12T09:30:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747065+0000", 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8770999999999997e-05 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:10993 3.2 3:47a13fed:::rbd_data.1191bbcdfbf6.0000000000000003:7 [read 10584064~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.626572913, 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033553980999999997, 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10993 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.713631+0000", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720607+0000", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069764909999999996 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720616+0000", 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2289999999999992e-06 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720631+0000", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4483e-05 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.720640+0000", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3749999999999992e-06 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747095+0000", 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026454688000000001 2024-09-12T09:30:09.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747133+0000", 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7719000000000003e-05 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.747185+0000", 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1996000000000002e-05 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:49995 3.15 e50/23 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head v 50'152, mlcod=49'150)", 2024-09-12T09:30:09.430 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7608807970000004, 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053471088999999999, 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:09.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.579323+0000", 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611150+0000", 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031827393000000002 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:09.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611191+0000", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0123000000000002e-05 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611215+0000", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4664999999999998e-05 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611349+0000", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013327799999999999 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:09.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.611373+0000", 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4176999999999999e-05 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.632751+0000", 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021378476 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.632794+0000", 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2546999999999998e-05 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:09.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:09.435 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:09.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:09.437 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:09.437+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:09.537 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:09.533+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:09.569 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3.b84de7ce (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.82149854600000005, 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.82154264099999996, 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:09.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:30:09.577 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:09.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:09.638 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:09.633+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:09.705 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:09.705 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:09.705 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:09.705 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:09.705 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:09.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:09.739 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:09.733+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:09.777 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone10 intact 2024-09-12T09:30:09.840 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:09.837+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:09.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.561422533, 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:09.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:09.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51587 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23298048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.237989+0000", 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6124853109999999, 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060241125999999999, 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51587 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.237989+0000", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.237989+0000", 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.237990+0000", 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7199999999999999e-07 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.237992+0000", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6680000000000001e-06 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.237999+0000", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4729999999999997e-06 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238003+0000", 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.297971+0000", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059967929000000003 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298030+0000", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8740999999999999e-05 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298230+0000", 2024-09-12T09:30:09.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020014499999999999 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51588 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23396352~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238011+0000", 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612462888, 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060743685999999998, 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51588 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238011+0000", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238011+0000", 2024-09-12T09:30:09.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238012+0000", 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238014+0000", 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8840000000000001e-06 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238020+0000", 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8749999999999997e-06 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238024+0000", 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5279999999999997e-06 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298251+0000", 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060227058999999999 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298310+0000", 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270999999999998e-05 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298755+0000", 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044563300000000002 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51589 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23494656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238030+0000", 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612444539, 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.061080701000000001, 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51589 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238030+0000", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238030+0000", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238030+0000", 2024-09-12T09:30:09.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238032+0000", 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238038+0000", 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5979999999999996e-06 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238043+0000", 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2129999999999999e-06 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298777+0000", 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.060734212000000003 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.298836+0000", 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8610999999999998e-05 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.299111+0000", 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027480899999999999 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51590 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23592960~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238049+0000", 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612425674, 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063066828000000005, 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.874 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51590 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238049+0000", 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238049+0000", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238049+0000", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4799999999999998e-07 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238059+0000", 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3179999999999996e-06 2024-09-12T09:30:09.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238069+0000", 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0149000000000001e-05 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238077+0000", 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.14e-06 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.300790+0000", 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.062713088 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.300854+0000", 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3919999999999998e-05 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301115+0000", 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026156500000000001 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51591 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23691264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238089+0000", 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6123851, 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063347935999999994, 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51591 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238089+0000", 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238089+0000", 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238090+0000", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238093+0000", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3680000000000002e-06 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238100+0000", 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9519999999999997e-06 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238105+0000", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301149+0000", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063043855999999995 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301206+0000", 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7290999999999997e-05 2024-09-12T09:30:09.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301437+0000", 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000230757 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51592 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23789568~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238112+0000", 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6123624009999999, 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063699290000000006, 2024-09-12T09:30:09.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51592 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238112+0000", 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238112+0000", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238112+0000", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238114+0000", 2024-09-12T09:30:09.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8220000000000001e-06 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238120+0000", 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4600000000000002e-06 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238124+0000", 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4920000000000004e-06 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301478+0000", 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063353722000000001 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301535+0000", 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7519000000000003e-05 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301811+0000", 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027584300000000001 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51593 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23887872~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238130+0000", 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612344298, 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063988022000000006, 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51593 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238130+0000", 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238130+0000", 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238130+0000", 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238132+0000", 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.686e-06 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238138+0000", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238144+0000", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7279999999999998e-06 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301842+0000", 2024-09-12T09:30:09.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063698041999999996 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.301898+0000", 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5948e-05 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302118+0000", 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021981400000000001 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51594 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 23986176~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238150+0000", 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612324018, 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.064334199999999994, 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51594 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238150+0000", 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238150+0000", 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238151+0000", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238153+0000", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.765e-06 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238158+0000", 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5500000000000002e-06 2024-09-12T09:30:09.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238168+0000", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9119999999999993e-06 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302149+0000", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.063981276000000004 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302205+0000", 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6056e-05 2024-09-12T09:30:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302485+0000", 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027918300000000001 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51595 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24084480~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238174+0000", 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612300098, 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.064617516, 2024-09-12T09:30:09.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51595 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238174+0000", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238174+0000", 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238175+0000", 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-07 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238182+0000", 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0040000000000002e-06 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238188+0000", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0240000000000003e-06 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238192+0000", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000003e-06 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302518+0000", 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.064325955000000004 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302576+0000", 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7901e-05 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302792+0000", 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021592599999999999 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51596 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24182784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238204+0000", 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612269991, 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.064922928000000005, 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51596 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238204+0000", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238204+0000", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238205+0000", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7800000000000002e-07 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238207+0000", 2024-09-12T09:30:09.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4219999999999999e-06 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238213+0000", 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8459999999999996e-06 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238218+0000", 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4580000000000003e-06 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302823+0000", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.06460523 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.302879+0000", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5995000000000002e-05 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303127+0000", 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024849900000000001 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51597 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24281088~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238223+0000", 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612250848, 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065203430000000007, 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51597 2024-09-12T09:30:09.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238223+0000", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238223+0000", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238224+0000", 2024-09-12T09:30:09.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238226+0000", 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5939999999999999e-06 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238233+0000", 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4919999999999996e-06 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238237+0000", 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.442e-06 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303158+0000", 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.064920810999999995 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303216+0000", 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7328999999999999e-05 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303427+0000", 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021134800000000001 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51598 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24379392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238243+0000", 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6122309129999999, 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065485452, 2024-09-12T09:30:09.910 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51598 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238243+0000", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238243+0000", 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238244+0000", 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2399999999999999e-07 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238246+0000", 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6640000000000001e-06 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238251+0000", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6760000000000004e-06 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238256+0000", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4660000000000001e-06 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303458+0000", 2024-09-12T09:30:09.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065202089000000005 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303514+0000", 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6375999999999997e-05 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303729+0000", 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021475699999999999 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51599 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24477696~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238263+0000", 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6122114220000001, 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065811172000000001, 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51599 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238263+0000", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238263+0000", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238264+0000", 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7800000000000001e-07 2024-09-12T09:30:09.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238266+0000", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6259999999999999e-06 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238277+0000", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0478e-05 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238283+0000", 2024-09-12T09:30:09.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7320000000000001e-06 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303760+0000", 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065476339999999994 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.303818+0000", 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7813999999999999e-05 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.304074+0000", 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025650400000000001 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51600 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24576000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238291+0000", 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6121837289999998, 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.066087213000000006, 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51600 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238291+0000", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238291+0000", 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238291+0000", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999996e-07 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238293+0000", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8619999999999999e-06 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238299+0000", 2024-09-12T09:30:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9220000000000002e-06 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238306+0000", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.304106+0000", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.065800111999999994 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.304162+0000", 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6301999999999997e-05 2024-09-12T09:30:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.304378+0000", 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000215429 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51601 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24674304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238317+0000", 2024-09-12T09:30:09.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6121574000000001, 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.067813006999999995, 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51601 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238317+0000", 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238317+0000", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238317+0000", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238319+0000", 2024-09-12T09:30:09.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9379999999999999e-06 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238326+0000", 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2439999999999998e-06 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238334+0000", 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2630000000000003e-06 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.305735+0000", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.067400917000000005 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.305798+0000", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3497000000000005e-05 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306130+0000", 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033164799999999998 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51602 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24772608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238345+0000", 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.6121297829999999, 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068240571999999999, 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51602 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238345+0000", 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238345+0000", 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238345+0000", 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238356+0000", 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1113e-05 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238370+0000", 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3134e-05 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238378+0000", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5229999999999995e-06 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306148+0000", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.067770160999999995 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306207+0000", 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8365000000000001e-05 2024-09-12T09:30:09.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306585+0000", 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037849200000000002 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51603 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24870912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238391+0000", 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612083449, 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068585894999999994, 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51603 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238391+0000", 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238391+0000", 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238392+0000", 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238395+0000", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3079999999999999e-06 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238407+0000", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1459e-05 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238416+0000", 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0520000000000007e-06 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306604+0000", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068188909000000006 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306665+0000", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0151000000000001e-05 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306977+0000", 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000312176 2024-09-12T09:30:09.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51604 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 24969216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238428+0000", 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612046455, 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068961050999999995, 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51604 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238428+0000", 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238428+0000", 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238429+0000", 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238432+0000", 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.394e-06 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238444+0000", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2121e-05 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238455+0000", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1146e-05 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.306996+0000", 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068540123999999994 2024-09-12T09:30:09.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.307052+0000", 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6558999999999998e-05 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.307389+0000", 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033679100000000002 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:51605 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:c [read 25067520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.238470+0000", 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.612004287, 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069293720000000003, 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 51605 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238470+0000", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238470+0000", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238471+0000", 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1999999999999998e-07 2024-09-12T09:30:09.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238474+0000", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.286e-06 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238488+0000", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3468999999999999e-05 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.238498+0000", 2024-09-12T09:30:09.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0234000000000001e-05 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.307408+0000", 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068909962000000005 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.307463+0000", 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-05 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:09.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.307764+0000", 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030070900000000002 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:09.946 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:09.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:09.948 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:09.944+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:10.021 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-12T09:30:10.049 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:10.045+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:10.123 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:10.123 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:10.123 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:10.123 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:10.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:10.149 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:10.145+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:10.250 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:10.245+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:10.258 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:10.258 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:10.258 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:10.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:10.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:10.335 INFO:teuthology.orchestra.run.smithi007.stdout:973 punch from 0x2177a7b to 0x217e64b, (0x6bd0 bytes) 2024-09-12T09:30:10.340 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.342 INFO:teuthology.orchestra.run.smithi007.stdout:977 trunc from 0x95635d5 to 0x7dc6f17 2024-09-12T09:30:10.351 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:10.344+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.011990091000001, 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:10.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:10.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.428 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.011894828999999, 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:10.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.008210205000001, 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.006405115, 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.005858440000001, 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:10.444 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:10.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.005089996999999, 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.004679629000002, 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:10.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:10.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.004540453000001, 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.460 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:10.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:10.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.464 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.996761641999999, 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:10.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.994397628000002, 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:10.470 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.983645735, 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.983449344, 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.981954924, 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44283 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 12943360~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.542152+0000", 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.879807005, 2024-09-12T09:30:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050355641999999999, 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44283 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542152+0000", 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542152+0000", 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542152+0000", 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542153+0000", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.257e-06 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542157+0000", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542160+0000", 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0319999999999999e-06 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592412+0000", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050251941000000001 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592453+0000", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1325000000000002e-05 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592507+0000", 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4591e-05 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44284 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 13017088~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.542163+0000", 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.879795664, 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050555656999999997, 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44284 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542163+0000", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542163+0000", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542163+0000", 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.36e-07 2024-09-12T09:30:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542165+0000", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.184e-06 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542168+0000", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1e-06 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542170+0000", 2024-09-12T09:30:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7439999999999999e-06 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592546+0000", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050375660000000003 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592626+0000", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0025999999999996e-05 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592719+0000", 2024-09-12T09:30:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2707000000000005e-05 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44285 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 12894208~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.542174+0000", 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.879784820999999, 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050789353000000002, 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44285 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542174+0000", 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542174+0000", 2024-09-12T09:30:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542174+0000", 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542176+0000", 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.133e-06 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542179+0000", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1049999999999999e-06 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542182+0000", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592759+0000", 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050577495 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592845+0000", 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5190000000000003e-05 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592963+0000", 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118801 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44286 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 12967936~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.542185+0000", 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.879773404, 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050904319000000003, 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44286 2024-09-12T09:30:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542185+0000", 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.542185+0000", 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.548816+0000", 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066303070000000002 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.548819+0000", 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.01e-06 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.548826+0000", 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5889999999999997e-06 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.548831+0000", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2409999999999997e-06 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.592996+0000", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044165779000000002 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.593035+0000", 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8741000000000001e-05 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.593090+0000", 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4651999999999998e-05 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44327 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 12906496~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.549368+0000", 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.87259091, 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050458144000000003, 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44327 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549368+0000", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549368+0000", 2024-09-12T09:30:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549368+0000", 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549373+0000", 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5220000000000001e-06 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549382+0000", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8939999999999999e-06 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549386+0000", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3839999999999999e-06 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599634+0000", 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050247474 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599716+0000", 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2033999999999994e-05 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599826+0000", 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110473 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44328 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 13033472~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.549390+0000", 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.872568442, 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050563721999999998, 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44328 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549390+0000", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549390+0000", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549391+0000", 2024-09-12T09:30:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549392+0000", 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549395+0000", 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.41e-06 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549398+0000", 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599858+0000", 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050459632999999997 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599897+0000", 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499000000000002e-05 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599954+0000", 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6641999999999997e-05 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:44329 3.16 3:69802ad3:::rbd_data.1191bbcdfbf6.0000000000000002:7 [read 13029376~4096] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:59.549402+0000", 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.872557036, 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050773812000000001, 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 44329 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549402+0000", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.549402+0000", 2024-09-12T09:30:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.556852+0000", 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074500210000000002 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.556859+0000", 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4819999999999997e-06 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.556867+0000", 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.345e-06 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.556871+0000", 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.599996+0000", 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043125278000000003 2024-09-12T09:30:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.600075+0000", 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8469999999999999e-05 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:59.600176+0000", 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100939 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:10.522 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:10.522 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:10.524 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:10.524 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:10.516+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:979 read 0x4f71489 thru 0x4f7f95f (0xe4d7 bytes) 2024-09-12T09:30:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:980 read 0x75e7934 thru 0x75eac0d (0x32da bytes) 2024-09-12T09:30:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:981 writesame 0x414184e thru 0x4143462 data_size 0xd(0x1c15 bytes) 2024-09-12T09:30:10.547 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:10.550 INFO:teuthology.orchestra.run.smithi007.stdout:982 write 0x76cc680 thru 0x76dbfbe (0xf93f bytes) 2024-09-12T09:30:10.552 INFO:teuthology.orchestra.run.smithi007.stdout:983 punch from 0x69e6349 to 0x69ef3f3, (0x90aa bytes) 2024-09-12T09:30:10.552 INFO:teuthology.orchestra.run.smithi007.stdout:984 write 0x78e64aa thru 0x78eaab4 (0x460b bytes) 2024-09-12T09:30:10.553 INFO:teuthology.orchestra.run.smithi007.stdout:985 write 0x4e0b506 thru 0x4e0ec38 (0x3733 bytes) 2024-09-12T09:30:10.563 INFO:teuthology.orchestra.run.smithi007.stdout:987 writesame 0x95336f0 thru 0x953de1c data_size 0x1(0xa72d bytes) 2024-09-12T09:30:10.563 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.563 INFO:teuthology.orchestra.run.smithi007.stdout:989 write 0x89450df thru 0x894a8c6 (0x57e8 bytes) 2024-09-12T09:30:10.568 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:10.571 INFO:teuthology.orchestra.run.smithi007.stdout:990 write 0x98c8e8e thru 0x98cb7d6 (0x2949 bytes) 2024-09-12T09:30:10.574 INFO:teuthology.orchestra.run.smithi007.stdout:991 read 0x88e15a3 thru 0x88edfaf (0xca0d bytes) 2024-09-12T09:30:10.575 INFO:teuthology.orchestra.run.smithi007.stdout:992 write 0x2db45ff thru 0x2db8395 (0x3d97 bytes) 2024-09-12T09:30:10.594 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:10.614 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:10.623 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:10.620+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:10.631 INFO:teuthology.orchestra.run.smithi007.stdout:994 trunc from 0x98cb7d7 to 0x111981f 2024-09-12T09:30:10.633 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:10.674 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:10.686 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [6, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:10.686 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:10.686 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-12T09:30:10.686 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.863352 2024-09-12T09:30:10.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.8633524197037025 2024-09-12T09:30:10.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:30:10.712 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:10.724 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:10.720+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:10.729 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:10.729 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:10.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:10.816 INFO:teuthology.orchestra.run.smithi007.stdout:995 write 0xcd4eabd thru 0xcd5d8d1 (0xee15 bytes) 2024-09-12T09:30:10.820 INFO:teuthology.orchestra.run.smithi007.stdout:998 read 0x607027 thru 0x6084f3 (0x14cd bytes) 2024-09-12T09:30:10.821 INFO:teuthology.orchestra.run.smithi007.stdout:999 writesame 0x9d103f8 thru 0x9d152bd data_size 0x1(0x4ec6 bytes) 2024-09-12T09:30:10.825 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:10.821+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:10.833 INFO:teuthology.orchestra.run.smithi007.stdout:1000 writesame 0xc8aaa75 thru 0xc8b2132 data_size 0x1(0x76be bytes) 2024-09-12T09:30:10.852 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.852 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.852 INFO:teuthology.orchestra.run.smithi007.stdout:1006 punch from 0x35f499d to 0x35f8a50, (0x40b3 bytes) 2024-09-12T09:30:10.852 INFO:teuthology.orchestra.run.smithi007.stdout:1007 punch from 0xa5962a to 0xa63b10, (0xa4e6 bytes) 2024-09-12T09:30:10.852 INFO:teuthology.orchestra.run.smithi007.stdout:1009 read 0x835ffc4 thru 0x836e359 (0xe396 bytes) 2024-09-12T09:30:10.870 INFO:teuthology.orchestra.run.smithi007.stdout:1011 trunc from 0xcd5d8d2 to 0x2fe36a 2024-09-12T09:30:10.906 INFO:teuthology.orchestra.run.smithi007.stdout:1012 trunc from 0x2fe36a to 0x727e638 2024-09-12T09:30:10.913 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:10.913 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:10.923 INFO:teuthology.orchestra.run.smithi007.stdout:1013 write 0xa924114 thru 0xa925d33 (0x1c20 bytes) 2024-09-12T09:30:10.926 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:10.921+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:10.927 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.927 INFO:teuthology.orchestra.run.smithi007.stdout:1015 read 0x16c5c87 thru 0x16d4912 (0xec8c bytes) 2024-09-12T09:30:10.927 INFO:teuthology.orchestra.run.smithi007.stdout:1017 read 0x2d4939c thru 0x2d4e2e3 (0x4f48 bytes) 2024-09-12T09:30:10.927 INFO:teuthology.orchestra.run.smithi007.stdout:1018 write 0x9f975a thru 0xa08b2d (0xf3d4 bytes) 2024-09-12T09:30:10.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:10.940 INFO:teuthology.orchestra.run.smithi007.stdout:1019 writesame 0xc5b19c5 thru 0xc5bc9f4 data_size 0x8(0xb030 bytes) 2024-09-12T09:30:10.940 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:10.941 INFO:teuthology.orchestra.run.smithi007.stdout:1021 write 0x97c43b3 thru 0x97c7835 (0x3483 bytes) 2024-09-12T09:30:10.944 INFO:teuthology.orchestra.run.smithi007.stdout:1022 trunc from 0xc5bc9f5 to 0xd238110 2024-09-12T09:30:10.949 INFO:teuthology.orchestra.run.smithi007.stdout:1023 read 0x4e6131 thru 0x4eb72b (0x55fb bytes) 2024-09-12T09:30:10.960 INFO:teuthology.orchestra.run.smithi007.stdout:1025 trunc from 0xd238110 to 0x506bb90 2024-09-12T09:30:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:1027 write 0xa467d7c thru 0xa46926c (0x14f1 bytes) 2024-09-12T09:30:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:1028 trunc from 0xa46926d to 0x7aa222e 2024-09-12T09:30:11.002 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.002 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.015 INFO:teuthology.orchestra.run.smithi007.stdout:1031 writesame 0xcfb6084 thru 0xcfbb67f data_size 0x2(0x55fc bytes) 2024-09-12T09:30:11.015 INFO:teuthology.orchestra.run.smithi007.stdout:1032 write 0xbfce2ac thru 0xbfcfd81 (0x1ad6 bytes) 2024-09-12T09:30:11.020 INFO:teuthology.orchestra.run.smithi007.stdout:1033 punch from 0x136faae to 0x137ea1e, (0xef70 bytes) 2024-09-12T09:30:11.027 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:11.020+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:1035 write 0xe29ccc6 thru 0xe2aae82 (0xe1bd bytes) 2024-09-12T09:30:11.046 INFO:teuthology.orchestra.run.smithi007.stdout:1037 trunc from 0xe2aae83 to 0x3920a66 2024-09-12T09:30:11.052 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.052 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:11.069 INFO:teuthology.orchestra.run.smithi007.stdout:1038 trunc from 0x3920a66 to 0x9351e59 2024-09-12T09:30:11.076 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.076 INFO:teuthology.orchestra.run.smithi007.stdout:1040 write 0x4663e50 thru 0x4664279 (0x42a bytes) 2024-09-12T09:30:11.080 INFO:teuthology.orchestra.run.smithi007.stdout:1041 punch from 0x83e66d2 to 0x83f5592, (0xeec0 bytes) 2024-09-12T09:30:11.092 INFO:teuthology.orchestra.run.smithi007.stdout:1042 write 0xd6386de thru 0xd639397 (0xcba bytes) 2024-09-12T09:30:11.095 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.095 INFO:teuthology.orchestra.run.smithi007.stdout:1044 read 0x9034294 thru 0x903b584 (0x72f1 bytes) 2024-09-12T09:30:11.095 INFO:teuthology.orchestra.run.smithi007.stdout:1045 punch from 0x9b100b0 to 0x9b19131, (0x9081 bytes) 2024-09-12T09:30:11.095 INFO:teuthology.orchestra.run.smithi007.stdout:1046 write 0xb4c8753 thru 0xb4d0404 (0x7cb2 bytes) 2024-09-12T09:30:11.098 INFO:teuthology.orchestra.run.smithi007.stdout:1048 write 0x85043d8 thru 0x8510324 (0xbf4d bytes) 2024-09-12T09:30:11.101 INFO:teuthology.orchestra.run.smithi007.stdout:1049 read 0x770cadf thru 0x7710c4e (0x4170 bytes) 2024-09-12T09:30:11.109 INFO:teuthology.orchestra.run.smithi007.stdout:1050 trunc from 0xd639398 to 0x764c747 2024-09-12T09:30:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.119 INFO:teuthology.orchestra.run.smithi007.stdout:1052 read 0x384cde8 thru 0x3854ffd (0x8216 bytes) 2024-09-12T09:30:11.120 INFO:teuthology.orchestra.run.smithi007.stdout:1054 read 0x2641b59 thru 0x264f4ff (0xd9a7 bytes) 2024-09-12T09:30:11.120 INFO:teuthology.orchestra.run.smithi007.stdout:1055 read 0x74b0264 thru 0x74bc1c3 (0xbf60 bytes) 2024-09-12T09:30:11.120 INFO:teuthology.orchestra.run.smithi007.stdout:1056 read 0x6fa9701 thru 0x6fab7da (0x20da bytes) 2024-09-12T09:30:11.127 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:11.125+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:11.134 INFO:teuthology.orchestra.run.smithi007.stdout:1057 writesame 0xe82c6e9 thru 0xe82f729 data_size 0x1(0x3041 bytes) 2024-09-12T09:30:11.134 INFO:teuthology.orchestra.run.smithi007.stdout:1058 punch from 0x5c07468 to 0x5c1328a, (0xbe22 bytes) 2024-09-12T09:30:11.134 INFO:teuthology.orchestra.run.smithi007.stdout:1059 write 0xbb0e24a thru 0xbb1cc7b (0xea32 bytes) 2024-09-12T09:30:11.137 INFO:teuthology.orchestra.run.smithi007.stdout:1061 write 0x1f6529 thru 0x1f7c31 (0x1709 bytes) 2024-09-12T09:30:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:1062 read 0xc1263ad thru 0xc12f09b (0x8cef bytes) 2024-09-12T09:30:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:1063 punch from 0x7d670cc to 0x7d6ef0d, (0x7e41 bytes) 2024-09-12T09:30:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:1064 punch from 0xdbd783b to 0xdbda27c, (0x2a41 bytes) 2024-09-12T09:30:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:1067 punch from 0xdb0de02 to 0xdb1673b, (0x8939 bytes) 2024-09-12T09:30:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:1070 write 0xe530605 thru 0xe5383a8 (0x7da4 bytes) 2024-09-12T09:30:11.143 INFO:teuthology.orchestra.run.smithi007.stdout:1071 write 0x40a40d8 thru 0x40a9c78 (0x5ba1 bytes) 2024-09-12T09:30:11.144 INFO:teuthology.orchestra.run.smithi007.stdout:1074 read 0x3a8171e thru 0x3a8f948 (0xe22b bytes) 2024-09-12T09:30:11.144 INFO:teuthology.orchestra.run.smithi007.stdout:1076 write 0x68a23bc thru 0x68a68f8 (0x453d bytes) 2024-09-12T09:30:11.148 INFO:teuthology.orchestra.run.smithi007.stdout:1078 write 0x78ecacc thru 0x78f041e (0x3953 bytes) 2024-09-12T09:30:11.152 INFO:teuthology.orchestra.run.smithi007.stdout:1079 read 0xf7b6ba thru 0xf874a8 (0xbdef bytes) 2024-09-12T09:30:11.152 INFO:teuthology.orchestra.run.smithi007.stdout:1080 writesame 0x30110df thru 0x3017425 data_size 0x55(0x6347 bytes) 2024-09-12T09:30:11.156 INFO:teuthology.orchestra.run.smithi007.stdout:1081 write 0xb6f54ae thru 0xb6fe7cc (0x931f bytes) 2024-09-12T09:30:11.160 INFO:teuthology.orchestra.run.smithi007.stdout:1082 read 0xbcfda0a thru 0xbd051b5 (0x77ac bytes) 2024-09-12T09:30:11.160 INFO:teuthology.orchestra.run.smithi007.stdout:1083 read 0x42aa84 thru 0x431dda (0x7357 bytes) 2024-09-12T09:30:11.180 INFO:teuthology.orchestra.run.smithi007.stdout:1084 trunc from 0xe82f72a to 0xb6bae6 2024-09-12T09:30:11.226 INFO:teuthology.orchestra.run.smithi007.stdout:1085 writesame 0x3661ac1 thru 0x366c5a4 data_size 0x2(0xaae4 bytes) 2024-09-12T09:30:11.228 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:11.224+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:11.229 INFO:teuthology.orchestra.run.smithi007.stdout:1086 write 0x729712 thru 0x737939 (0xe228 bytes) 2024-09-12T09:30:11.235 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.236 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.236 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.251 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:11.256 INFO:teuthology.orchestra.run.smithi007.stdout:1089 writesame 0x7dabb20 thru 0x7db391e data_size 0x1(0x7dff bytes) 2024-09-12T09:30:11.256 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.257 INFO:teuthology.orchestra.run.smithi007.stdout:1091 read 0x728f3a0 thru 0x729237b (0x2fdc bytes) 2024-09-12T09:30:11.268 INFO:teuthology.orchestra.run.smithi007.stdout:1092 write 0x88484cb thru 0x88557a9 (0xd2df bytes) 2024-09-12T09:30:11.288 INFO:teuthology.orchestra.run.smithi007.stdout:1093 trunc from 0x88557aa to 0xeb19c93 2024-09-12T09:30:11.300 INFO:teuthology.orchestra.run.smithi007.stdout:1094 punch from 0x54b2afc to 0x54bd40b, (0xa90f bytes) 2024-09-12T09:30:11.300 INFO:teuthology.orchestra.run.smithi007.stdout:1095 write 0xc0da03d thru 0xc0e14b1 (0x7475 bytes) 2024-09-12T09:30:11.307 INFO:teuthology.orchestra.run.smithi007.stdout:1096 writesame 0x1f644b4 thru 0x1f73360 data_size 0x5d(0xeead bytes) 2024-09-12T09:30:11.314 INFO:teuthology.orchestra.run.smithi007.stdout:1097 punch from 0x9a8be03 to 0x9a8d13a, (0x1337 bytes) 2024-09-12T09:30:11.314 INFO:teuthology.orchestra.run.smithi007.stdout:1099 read 0x884e59e thru 0x884f04e (0xab1 bytes) 2024-09-12T09:30:11.315 INFO:teuthology.orchestra.run.smithi007.stdout:1101 punch from 0xa38b44a to 0xa3984a0, (0xd056 bytes) 2024-09-12T09:30:11.316 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.316 INFO:teuthology.orchestra.run.smithi007.stdout:1103 read 0x15682da thru 0x1573a3e (0xb765 bytes) 2024-09-12T09:30:11.316 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.316 INFO:teuthology.orchestra.run.smithi007.stdout:1106 write 0x8bbece1 thru 0x8bc10c0 (0x23e0 bytes) 2024-09-12T09:30:11.324 INFO:teuthology.orchestra.run.smithi007.stdout:1107 read 0xa760ef7 thru 0xa76ee08 (0xdf12 bytes) 2024-09-12T09:30:11.324 INFO:teuthology.orchestra.run.smithi007.stdout:1108 read 0x3e520ce thru 0x3e5910d (0x7040 bytes) 2024-09-12T09:30:11.324 INFO:teuthology.orchestra.run.smithi007.stdout:1109 write 0xdb69aab thru 0xdb6e322 (0x4878 bytes) 2024-09-12T09:30:11.332 INFO:teuthology.orchestra.run.smithi007.stdout:1111 punch from 0xe79a769 to 0xe7a32b7, (0x8b4e bytes) 2024-09-12T09:30:11.332 INFO:teuthology.orchestra.run.smithi007.stdout:1112 punch from 0x8824358 to 0x882b662, (0x730a bytes) 2024-09-12T09:30:11.332 INFO:teuthology.orchestra.run.smithi007.stdout:1113 read 0x91156cb thru 0x91232f5 (0xdc2b bytes) 2024-09-12T09:30:11.333 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.333 INFO:teuthology.orchestra.run.smithi007.stdout:1116 punch from 0xa460821 to 0xa466f6a, (0x6749 bytes) 2024-09-12T09:30:11.333 INFO:teuthology.orchestra.run.smithi007.stdout:1118 writesame 0xe348b0a thru 0xe34d055 data_size 0x5(0x454c bytes) 2024-09-12T09:30:11.343 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:11.340+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:11.351 INFO:teuthology.orchestra.run.smithi007.stdout:1119 trunc from 0xeb19c93 to 0xaf30d4a 2024-09-12T09:30:11.377 INFO:teuthology.orchestra.run.smithi007.stdout:1121 read 0xdd69b6 thru 0xde0443 (0x9a8e bytes) 2024-09-12T09:30:11.378 INFO:teuthology.orchestra.run.smithi007.stdout:1123 write 0x653d5f2 thru 0x6549db1 (0xc7c0 bytes) 2024-09-12T09:30:11.382 INFO:teuthology.orchestra.run.smithi007.stdout:1124 write 0x7603c04 thru 0x760ff0d (0xc30a bytes) 2024-09-12T09:30:11.385 INFO:teuthology.orchestra.run.smithi007.stdout:1125 read 0x976e349 thru 0x97740da (0x5d92 bytes) 2024-09-12T09:30:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:1126 write 0xc6e4369 thru 0xc6ec551 (0x81e9 bytes) 2024-09-12T09:30:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:1127 write 0x716f9c3 thru 0x717f3f3 (0xfa31 bytes) 2024-09-12T09:30:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:1128 write 0x2a7a1fe thru 0x2a86d77 (0xcb7a bytes) 2024-09-12T09:30:11.421 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.421 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.422 INFO:teuthology.orchestra.run.smithi007.stdout:1130 read 0x54a834d thru 0x54b078f (0x8443 bytes) 2024-09-12T09:30:11.430 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:11.425+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:11.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:11.439 INFO:teuthology.orchestra.run.smithi007.stdout:1131 write 0xd5b7621 thru 0xd5bd15e (0x5b3e bytes) 2024-09-12T09:30:11.446 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.446 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.446 INFO:teuthology.orchestra.run.smithi007.stdout:1136 punch from 0x9562140 to 0x95643fa, (0x22ba bytes) 2024-09-12T09:30:11.446 INFO:teuthology.orchestra.run.smithi007.stdout:1140 write 0x504242e thru 0x5044e7e (0x2a51 bytes) 2024-09-12T09:30:11.452 INFO:teuthology.orchestra.run.smithi007.stdout:1141 punch from 0x349cb44 to 0x34a4bd4, (0x8090 bytes) 2024-09-12T09:30:11.475 INFO:teuthology.orchestra.run.smithi007.stdout:1142 trunc from 0xd5bd15f to 0x53d87d2 2024-09-12T09:30:11.510 INFO:teuthology.orchestra.run.smithi007.stdout:1143 writesame 0x3752288 thru 0x37584b1 data_size 0x23(0x622a bytes) 2024-09-12T09:30:11.527 INFO:teuthology.orchestra.run.smithi007.stdout:1144 trunc from 0x53d87d2 to 0x353dfe 2024-09-12T09:30:11.531 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:11.525+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:11.571 INFO:teuthology.orchestra.run.smithi007.stdout:1149 read 0x259bd6 thru 0x25a222 (0x64d bytes) 2024-09-12T09:30:11.573 INFO:teuthology.orchestra.run.smithi007.stdout:1150 punch from 0x1906d4 to 0x19d2af, (0xcbdb bytes) 2024-09-12T09:30:11.573 INFO:teuthology.orchestra.run.smithi007.stdout:1151 read 0x1d8186 thru 0x1e5c6c (0xdae7 bytes) 2024-09-12T09:30:11.574 INFO:teuthology.orchestra.run.smithi007.stdout:1152 read 0x2f706c thru 0x2f995b (0x28f0 bytes) 2024-09-12T09:30:11.575 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.597 INFO:teuthology.orchestra.run.smithi007.stdout:1155 write 0x3a19eae thru 0x3a24657 (0xa7aa bytes) 2024-09-12T09:30:11.605 INFO:teuthology.orchestra.run.smithi007.stdout:1157 read 0x2a1d3c1 thru 0x2a2c3b9 (0xeff9 bytes) 2024-09-12T09:30:11.607 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.607 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:1159 writesame 0x40b9c16 thru 0x40bb0c1 data_size 0x24(0x14ac bytes) 2024-09-12T09:30:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:1160 read 0x250ba1d thru 0x250fae3 (0x40c7 bytes) 2024-09-12T09:30:11.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:1161 write 0x36a9958 thru 0x36b7598 (0xdc41 bytes) 2024-09-12T09:30:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:1162 writesame 0xebe11f6 thru 0xebe222c data_size 0x7(0x1037 bytes) 2024-09-12T09:30:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:1163 read 0xbe89a3b thru 0xbe90ec8 (0x748e bytes) 2024-09-12T09:30:11.661 INFO:teuthology.orchestra.run.smithi007.stdout:1164 write 0xcacfc6f thru 0xcadad1e (0xb0b0 bytes) 2024-09-12T09:30:11.662 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:11.657+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:11.666 INFO:teuthology.orchestra.run.smithi007.stdout:1165 read 0x8426426 thru 0x842901d (0x2bf8 bytes) 2024-09-12T09:30:11.666 INFO:teuthology.orchestra.run.smithi007.stdout:1166 read 0x15d6cb5 thru 0x15da260 (0x35ac bytes) 2024-09-12T09:30:11.667 INFO:teuthology.orchestra.run.smithi007.stdout:1167 read 0x91d2e9c thru 0x91d3011 (0x176 bytes) 2024-09-12T09:30:11.667 INFO:teuthology.orchestra.run.smithi007.stdout:1168 write 0x3665743 thru 0x36717e4 (0xc0a2 bytes) 2024-09-12T09:30:11.670 INFO:teuthology.orchestra.run.smithi007.stdout:1169 read 0xe9c4517 thru 0xe9cae01 (0x68eb bytes) 2024-09-12T09:30:11.670 INFO:teuthology.orchestra.run.smithi007.stdout:1173 read 0xd86aebe thru 0xd870c87 (0x5dca bytes) 2024-09-12T09:30:11.670 INFO:teuthology.orchestra.run.smithi007.stdout:1176 write 0xb9f8838 thru 0xba00b07 (0x82d0 bytes) 2024-09-12T09:30:11.675 INFO:teuthology.orchestra.run.smithi007.stdout:1177 writesame 0xccf4ca0 thru 0xccf8086 data_size 0x2b(0x33e7 bytes) 2024-09-12T09:30:11.677 INFO:teuthology.orchestra.run.smithi007.stdout:1178 punch from 0x834cf67 to 0x834fd53, (0x2dec bytes) 2024-09-12T09:30:11.677 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.677 INFO:teuthology.orchestra.run.smithi007.stdout:1181 read 0xeb0599f thru 0xeb0d10f (0x7771 bytes) 2024-09-12T09:30:11.677 INFO:teuthology.orchestra.run.smithi007.stdout:1182 writesame 0xa59412f thru 0xa59cc17 data_size 0x1(0x8ae9 bytes) 2024-09-12T09:30:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 0.863352 (8565802) 2024-09-12T09:30:11.706 INFO:teuthology.orchestra.run.smithi007.stdout:1183 writesame 0x5d290a8 thru 0x5d2debb data_size 0x26(0x4e14 bytes) 2024-09-12T09:30:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:30:11.718 INFO:teuthology.orchestra.run.smithi007.stdout:1184 write 0x3bb042e thru 0x3bbbc87 (0xb85a bytes) 2024-09-12T09:30:11.721 INFO:teuthology.orchestra.run.smithi007.stdout:1185 write 0x12f2a83 thru 0x12fed72 (0xc2f0 bytes) 2024-09-12T09:30:11.725 INFO:teuthology.orchestra.run.smithi007.stdout:1187 write 0x8257963 thru 0x82608cd (0x8f6b bytes) 2024-09-12T09:30:11.732 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:11.729+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:11.748 INFO:teuthology.orchestra.run.smithi007.stdout:1188 trunc from 0xebe222d to 0x229c30d 2024-09-12T09:30:11.765 INFO:teuthology.orchestra.run.smithi007.stdout:1189 writesame 0x5ffbc8 thru 0x6038ac data_size 0x77(0x3ce5 bytes) 2024-09-12T09:30:11.769 INFO:teuthology.orchestra.run.smithi007.stdout:1190 read 0x1672ec1 thru 0x1679b4a (0x6c8a bytes) 2024-09-12T09:30:11.781 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.781 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.784 INFO:teuthology.orchestra.run.smithi007.stdout:1191 write 0x7b6f278 thru 0x7b7230f (0x3098 bytes) 2024-09-12T09:30:11.789 INFO:teuthology.orchestra.run.smithi007.stdout:1192 read 0x36b07b thru 0x376982 (0xb908 bytes) 2024-09-12T09:30:11.789 INFO:teuthology.orchestra.run.smithi007.stdout:1193 punch from 0x1373298 to 0x1375a61, (0x27c9 bytes) 2024-09-12T09:30:11.789 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.789 INFO:teuthology.orchestra.run.smithi007.stdout:1196 punch from 0x4a2aa95 to 0x4a343d8, (0x9943 bytes) 2024-09-12T09:30:11.789 INFO:teuthology.orchestra.run.smithi007.stdout:1198 writesame 0x591f06f thru 0x5921b92 data_size 0x4(0x2b24 bytes) 2024-09-12T09:30:11.796 INFO:teuthology.orchestra.run.smithi007.stdout:1199 read 0x4f775dc thru 0x4f84b8c (0xd5b1 bytes) 2024-09-12T09:30:11.796 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:11.796 INFO:teuthology.orchestra.run.smithi007.stdout:1203 write 0x2c5bc8c thru 0x2c5f2aa (0x361f bytes) 2024-09-12T09:30:11.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:11.799 INFO:teuthology.orchestra.run.smithi007.stdout:1204 read 0x1b03618 thru 0x1b0a482 (0x6e6b bytes) 2024-09-12T09:30:11.800 INFO:teuthology.orchestra.run.smithi007.stdout:1207 punch from 0x65fae8f to 0x6606c3e, (0xbdaf bytes) 2024-09-12T09:30:11.801 INFO:teuthology.orchestra.run.smithi007.stdout:1208 trunc from 0x7b72310 to 0x8c22fe8 2024-09-12T09:30:11.807 INFO:teuthology.orchestra.run.smithi007.stdout:1209 read 0x6a7bc61 thru 0x6a895da (0xd97a bytes) 2024-09-12T09:30:11.808 INFO:teuthology.orchestra.run.smithi007.stdout:1210 read 0xb2cae4 thru 0xb3312d (0x664a bytes) 2024-09-12T09:30:11.808 INFO:teuthology.orchestra.run.smithi007.stdout:1212 punch from 0x837b67a to 0x8388288, (0xcc0e bytes) 2024-09-12T09:30:11.808 INFO:teuthology.orchestra.run.smithi007.stdout:1214 write 0x2d68eac thru 0x2d6b525 (0x267a bytes) 2024-09-12T09:30:11.810 INFO:teuthology.orchestra.run.smithi007.stdout:1215 write 0x21bcb34 thru 0x21c4376 (0x7843 bytes) 2024-09-12T09:30:11.814 INFO:teuthology.orchestra.run.smithi007.stdout:1216 read 0x576cff0 thru 0x577c4d0 (0xf4e1 bytes) 2024-09-12T09:30:11.815 INFO:teuthology.orchestra.run.smithi007.stdout:1217 read 0x2350fb1 thru 0x235bace (0xab1e bytes) 2024-09-12T09:30:11.815 INFO:teuthology.orchestra.run.smithi007.stdout:1220 clone 12 order 21 su 8192 sc 11 2024-09-12T09:30:11.833 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:11.828+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:11.916 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:11.916 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:11.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:11.972 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:11.969+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:12.034 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:12.029+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:12.097 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:12.097 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:12.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:12.135 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:12.132+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:12.236 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:12.233+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:12.284 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:12.285 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:12.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:12.337 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:12.332+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:12.438 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:12.433+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:12.460 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:12.460 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:12.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:12.539 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:12.532+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:12.640 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:12.636+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:12.645 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:12.645 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:12.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:12.740 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:12.736+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:12.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:12.797 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:12.797 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:12.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:12.841 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:12.836+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:12.852 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone11 2024-09-12T09:30:12.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:12.926 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:12.942 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:12.937+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:12.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:12.945 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:12.952 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:12.952 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:12.965 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:12.965 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:12.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:12.987 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:12.987 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:13.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:13.012 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-12T09:30:13.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:13.043 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:13.036+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:13.052 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:13.144 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:13.141+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:13.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1037 3.0 e28/23 3:03bced3e:::rbd_data.1191192d73a0.0000000000000022:head v 28'12, mlcod=28'10)", 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.693415121999998, 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001724747, 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465479+0000", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465482+0000", 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7970000000000001e-06 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465540+0000", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7562999999999998e-05 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465584+0000", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4653000000000001e-05 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465608+0000", 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3566000000000001e-05 2024-09-12T09:30:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465769+0000", 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161652 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.465810+0000", 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0256999999999998e-05 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467125+0000", 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00131514 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.467204+0000", 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9118999999999999e-05 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1060 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'5, mlcod=28'3)", 2024-09-12T09:30:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.628751497000003, 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021205709999999999, 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530143+0000", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530145+0000", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8530000000000001e-06 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530243+0000", 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7979000000000005e-05 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530271+0000", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8263999999999999e-05 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530284+0000", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.327e-05 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530430+0000", 2024-09-12T09:30:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000145632 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.530497+0000", 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7361999999999996e-05 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532149+0000", 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016521750000000001 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.532263+0000", 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114036 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.624182615000002, 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.446521203000003, 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:30:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:30:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1134 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'15, mlcod=28'13)", 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.423562838000002, 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017480740000000001, 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735331+0000", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735334+0000", 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4770000000000002e-06 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735349+0000", 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735388+0000", 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8707000000000001e-05 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735402+0000", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4696e-05 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735640+0000", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023778800000000001 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.735740+0000", 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100049 2024-09-12T09:30:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.736984+0000", 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012437450000000001 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.737080+0000", 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5679000000000003e-05 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:2247 3.0 e33/23 3:04eb8707:::rbd_data.1191d634a7ff.0000000000000004:head v 33'31, mlcod=31'28)", 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.993214419000001, 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015296330000000001, 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165680+0000", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165687+0000", 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165706+0000", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8895000000000001e-05 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165717+0000", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0572e-05 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165798+0000", 2024-09-12T09:30:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1468e-05 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.165823+0000", 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4595000000000001e-05 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167163+0000", 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013407569999999999 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.167210+0000", 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6099 3.9 e33/23 3:914773c0:::rbd_data.11912e4e65e9.0000000000000062:head v 33'7, mlcod=28'5)", 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.364441098, 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023702649999999999, 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794453+0000", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794454+0000", 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794889+0000", 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00043553500000000002 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794905+0000", 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6144e-05 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.794914+0000", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8969999999999997e-06 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795008+0000", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3913999999999996e-05 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795028+0000", 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9468999999999999e-05 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796777+0000", 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017492899999999999 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.796823+0000", 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6591000000000002e-05 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:6128 3.9 e33/23 3:95fc6e92:::rbd_data.11912e4e65e9.0000000000000063:head v 33'8, mlcod=28'5)", 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.363665466, 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001886235, 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795229+0000", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795230+0000", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0070000000000001e-06 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795479+0000", 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024871000000000002 2024-09-12T09:30:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795493+0000", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4793999999999999e-05 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795501+0000", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0269999999999995e-06 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795659+0000", 2024-09-12T09:30:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015762299999999999 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.795671+0000", 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797075+0000", 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014034340000000001 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:45.797115+0000", 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0547000000000003e-05 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.027749356000001, 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.020940678999999, 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:30:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:30:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:30:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10625 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'15, mlcod=34'13)", 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.631993113, 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020725600000000002, 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526901+0000", 2024-09-12T09:30:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.526903+0000", 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.776e-06 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527721+0000", 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081786300000000005 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527750+0000", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8649e-05 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527770+0000", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0015999999999999e-05 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527852+0000", 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2396000000000002e-05 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.527869+0000", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6982000000000001e-05 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528885+0000", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001016227 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.528974+0000", 2024-09-12T09:30:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8651000000000002e-05 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:10623 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'6, mlcod=33'4)", 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.628623080000001, 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014836669999999999, 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530271+0000", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530273+0000", 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.263e-06 2024-09-12T09:30:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530299+0000", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6156000000000002e-05 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530317+0000", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8488999999999999e-05 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530327+0000", 2024-09-12T09:30:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7669999999999998e-06 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530434+0000", 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010745 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.530468+0000", 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3649999999999998e-05 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531650+0000", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011823960000000001 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.531755+0000", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104496 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.123003285999999, 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:30:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:30:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11694 3.9 e38/23 3:90310728:::rbd_data.11913fee8ce1.000000000000002a:head v 38'17, mlcod=38'15)", 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.119715792000001, 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015297240000000001, 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039179+0000", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999999e-07 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039276+0000", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6244999999999998e-05 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039292+0000", 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-12T09:30:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039302+0000", 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9369999999999999e-06 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039368+0000", 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5711999999999997e-05 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.039387+0000", 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9386999999999999e-05 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040592+0000", 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012047270000000001 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.040708+0000", 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116242 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.14 53/52 [PushOp(3:2cab5d4e:::rbd_data.11914330b43e.0000000000000013:9, version: 45'206, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:2cab5d4e:::rbd_data.11914330b43e.0000000000000013:9@45'206, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.0389099140000004, 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001534037, 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119984+0000", 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.119985+0000", 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5099999999999999e-07 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120704+0000", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071853100000000001 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120730+0000", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6276999999999999e-05 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120756+0000", 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5981000000000001e-05 2024-09-12T09:30:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120840+0000", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3794000000000004e-05 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.120849+0000", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5189999999999998e-06 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.121518+0000", 2024-09-12T09:30:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066918399999999997 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8594803439999996, 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:30:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51963 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'64, mlcod=49'62)", 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7738338379999998, 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001823409, 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385060+0000", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385062+0000", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3400000000000001e-06 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385070+0000", 2024-09-12T09:30:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7789999999999996e-06 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385089+0000", 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9561000000000001e-05 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385099+0000", 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385166+0000", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.385199+0000", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2799999999999998e-05 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.386852+0000", 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016533749999999999 2024-09-12T09:30:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.386884+0000", 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609000000000002e-05 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:52727 3.1e e55/46 3:7b8b3a94:::rbd_header.119172280d2a:head v 55'55, mlcod=53'53)", 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.020161+0000", 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.138733255, 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001516753, 2024-09-12T09:30:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020161+0000", 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020161+0000", 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020164+0000", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020178+0000", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4783000000000001e-05 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020216+0000", 2024-09-12T09:30:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7642000000000003e-05 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020234+0000", 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8088999999999999e-05 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020375+0000", 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014093999999999999 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.020415+0000", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0302000000000001e-05 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.021596+0000", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011806099999999999 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.021678+0000", 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1834000000000003e-05 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.6770614259999999, 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54570 3.9 e55/23 3:9606e935:::rbd_data.1191175d8602.000000000000000c:head v 55'55, mlcod=49'52)", 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.6657638070000003, 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037370770000000001, 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493133+0000", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3329999999999999e-06 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493160+0000", 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6934e-05 2024-09-12T09:30:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493178+0000", 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7972e-05 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493191+0000", 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2773e-05 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.495621+0000", 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024305020000000002 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.495649+0000", 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8350000000000001e-05 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496822+0000", 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011723059999999999 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496868+0000", 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5906999999999999e-05 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.251 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:13.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:13.253 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:13.249+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:13.353 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:13.349+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:13.422 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:13.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:13.480 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:13.477+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:13.528 INFO:teuthology.orchestra.run.smithi007.stdout:1221 punch from 0x5a3aae0 to 0x5a4124a, (0x676a bytes) 2024-09-12T09:30:13.535 INFO:teuthology.orchestra.run.smithi007.stdout:1222 punch from 0xbb259 to 0xbb365, (0x10c bytes) 2024-09-12T09:30:13.545 INFO:teuthology.orchestra.run.smithi007.stdout:1223 write 0x9ceaea0 thru 0x9cf2763 (0x78c4 bytes) 2024-09-12T09:30:13.555 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:13.549+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:13.558 INFO:teuthology.orchestra.run.smithi007.stdout:1224 writesame 0x1bb4544 thru 0x1bbc749 data_size 0x22(0x8206 bytes) 2024-09-12T09:30:13.617 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:13.618 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:13.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:13.658 INFO:teuthology.orchestra.run.smithi007.stdout:1225 trunc from 0x9cf2764 to 0x498936b 2024-09-12T09:30:13.658 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:13.653+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:13.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:30:13.756 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:13.752+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:13.773 INFO:teuthology.orchestra.run.smithi007.stdout:1226 write 0x11ad56c thru 0x11bb4c4 (0xdf59 bytes) 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53986 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 14909440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3958232669999999, 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055388721000000002, 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53986 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387069+0000", 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.905e-06 2024-09-12T09:30:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387076+0000", 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387087+0000", 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1499e-05 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442148+0000", 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055060410999999997 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442229+0000", 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1441999999999997e-05 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442456+0000", 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000226534 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53987 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15007744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.395794864, 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055709171000000002, 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53987 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387096+0000", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387098+0000", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0770000000000002e-06 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387105+0000", 2024-09-12T09:30:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7619999999999998e-06 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387113+0000", 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4959999999999996e-06 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442489+0000", 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055376140999999997 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442610+0000", 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120234 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442805+0000", 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019502899999999999 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53988 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15106048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.395769472, 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057056229, 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53988 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387123+0000", 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387130+0000", 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5810000000000002e-06 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387135+0000", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1490000000000002e-06 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442836+0000", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055700889000000003 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442918+0000", 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000004e-05 2024-09-12T09:30:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.444177+0000", 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012590570000000001 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54006 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17367040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3955630540000001, 2024-09-12T09:30:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056089721000000002, 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54006 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389586+0000", 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002258552 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389632+0000", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5796e-05 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389644+0000", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2699e-05 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389667+0000", 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3076999999999999e-05 2024-09-12T09:30:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443194+0000", 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053526642999999999 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443320+0000", 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012563599999999999 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443417+0000", 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7318000000000006e-05 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54034 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17694720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3918002889999999, 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055444460000000001, 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54034 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391755+0000", 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066475499999999995 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391760+0000", 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.181e-06 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391904+0000", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014413300000000001 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391916+0000", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1507e-05 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446235+0000", 2024-09-12T09:30:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054319522000000002 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446313+0000", 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7745999999999997e-05 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446534+0000", 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221616 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54051 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20250624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3901293020000001, 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055664077999999999, 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54051 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392762+0000", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392764+0000", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4710000000000001e-06 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392772+0000", 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6969999999999993e-06 2024-09-12T09:30:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392777+0000", 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.587e-06 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447587+0000", 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.05480952 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447661+0000", 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4635000000000006e-05 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448425+0000", 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076352799999999997 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54052 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20348928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.3901025809999998, 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056119461000000002, 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54052 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393045+0000", 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025747600000000003 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393054+0000", 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0159999999999997e-06 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393064+0000", 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4050000000000006e-06 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393071+0000", 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6290000000000001e-06 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448481+0000", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055409753999999999 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448576+0000", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4969999999999994e-05 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448907+0000", 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033121099999999998 2024-09-12T09:30:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61223 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13336576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54330885600000001, 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055340981999999997, 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61223 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239582+0000", 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3500000000000002e-07 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239584+0000", 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3860000000000001e-06 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239635+0000", 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0985000000000001e-05 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239643+0000", 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8869999999999992e-06 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294604+0000", 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054961083000000001 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294717+0000", 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011312 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294922+0000", 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020508599999999999 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61227 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13533184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54263678299999996, 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055556323999999997, 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61227 2024-09-12T09:30:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240254+0000", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240257+0000", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.762e-06 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240265+0000", 2024-09-12T09:30:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8469999999999996e-06 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240271+0000", 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0689999999999998e-06 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295548+0000", 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055277706000000003 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295630+0000", 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1450000000000001e-05 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295810+0000", 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017996399999999999 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61229 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13631488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54261020599999998, 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055868963000000001, 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61229 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240281+0000", 2024-09-12T09:30:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240283+0000", 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240290+0000", 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0570000000000004e-06 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240295+0000", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.164e-06 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295854+0000", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055558626999999999 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295952+0000", 2024-09-12T09:30:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8405000000000004e-05 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296149+0000", 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019671099999999999 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61231 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13729792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54258610500000004, 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056138067999999999, 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61231 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240305+0000", 2024-09-12T09:30:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240307+0000", 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.294e-06 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240315+0000", 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1019999999999997e-06 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240321+0000", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9370000000000001e-06 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296180+0000", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055859345999999997 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296248+0000", 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7596999999999997e-05 2024-09-12T09:30:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296442+0000", 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019433699999999999 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61233 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13828096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54256134, 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056534344, 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61233 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240332+0000", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240340+0000", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240346+0000", 2024-09-12T09:30:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296492+0000", 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056146433000000003 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296606+0000", 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011335799999999999 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296863+0000", 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000257612 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61235 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13926400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54253482500000005, 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056785587999999998, 2024-09-12T09:30:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61235 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240356+0000", 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240358+0000", 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5390000000000002e-06 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240367+0000", 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6750000000000008e-06 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240372+0000", 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2789999999999998e-06 2024-09-12T09:30:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296882+0000", 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056509707999999999 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296969+0000", 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7366000000000003e-05 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297141+0000", 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017161100000000001 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61237 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14024704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54251037300000005, 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057667507, 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61237 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9400000000000001e-07 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240382+0000", 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1900000000000002e-06 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240389+0000", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4060000000000002e-06 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240394+0000", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9300000000000002e-06 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297172+0000", 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056777785999999997 2024-09-12T09:30:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297313+0000", 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014155099999999999 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298047+0000", 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073424999999999999 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61239 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54248823099999999, 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057981773, 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61239 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9000000000000002e-07 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240405+0000", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240412+0000", 2024-09-12T09:30:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.469e-06 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240427+0000", 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5102e-05 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298080+0000", 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057653049999999997 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298166+0000", 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5296000000000005e-05 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298384+0000", 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000218027 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61241 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14221312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54245222999999998, 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058244112000000001, 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61241 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240441+0000", 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7020000000000001e-06 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240450+0000", 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240455+0000", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298403+0000", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057947945000000001 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298473+0000", 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9863999999999994e-05 2024-09-12T09:30:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298682+0000", 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020926199999999999 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61243 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14319616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54242722799999998, 2024-09-12T09:30:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058591126, 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61243 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240466+0000", 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7010000000000001e-06 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240473+0000", 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240479+0000", 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.327e-06 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298722+0000", 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058243504000000001 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298783+0000", 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1137999999999996e-05 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299054+0000", 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027089799999999999 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61245 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14417920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54240345499999998, 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058955593000000001, 2024-09-12T09:30:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61245 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240494+0000", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5420000000000002e-06 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240511+0000", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240518+0000", 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5989999999999998e-06 2024-09-12T09:30:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299110+0000", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058592712999999998 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299190+0000", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9622000000000004e-05 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299442+0000", 2024-09-12T09:30:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025237399999999998 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61247 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14516224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54236278100000002, 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059307199999999997, 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61247 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240528+0000", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240531+0000", 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.615e-06 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240539+0000", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240545+0000", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0310000000000004e-06 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299503+0000", 2024-09-12T09:30:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058957756 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299625+0000", 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012207099999999999 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299835+0000", 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209605 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61249 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14614528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.54233649500000003, 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059607902999999997, 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61249 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240557+0000", 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5780000000000001e-06 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240565+0000", 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5040000000000002e-06 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240570+0000", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0690000000000001e-06 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299884+0000", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059313728000000003 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299967+0000", 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3404000000000003e-05 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.300162+0000", 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019417999999999999 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:13.880 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:13.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:13.882 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:13.876+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:13.917 INFO:teuthology.orchestra.run.smithi007.stdout:1228 writesame 0x5db153d thru 0x5dbf7be data_size 0x4f(0xe282 bytes) 2024-09-12T09:30:13.918 INFO:teuthology.orchestra.run.smithi007.stdout:1229 read 0x57adb15 thru 0x57bda53 (0xff3f bytes) 2024-09-12T09:30:13.918 INFO:teuthology.orchestra.run.smithi007.stdout:1230 punch from 0x2de4db7 to 0x2dec6b8, (0x7901 bytes) 2024-09-12T09:30:13.925 INFO:teuthology.orchestra.run.smithi007.stdout:1231 write 0x738a537 thru 0x739945c (0xef26 bytes) 2024-09-12T09:30:13.951 INFO:teuthology.orchestra.run.smithi007.stdout:1232 write 0xa3573b0 thru 0xa35e0f7 (0x6d48 bytes) 2024-09-12T09:30:13.959 INFO:teuthology.orchestra.run.smithi007.stdout:1234 writesame 0x7c0588b thru 0x7c0c4dd data_size 0x1(0x6c53 bytes) 2024-09-12T09:30:13.977 INFO:teuthology.orchestra.run.smithi007.stdout:1235 write 0xa993e6a thru 0xa99b7dd (0x7974 bytes) 2024-09-12T09:30:13.982 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:13.976+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:13.983 INFO:teuthology.orchestra.run.smithi007.stdout:1236 read 0x302d54b thru 0x30340f1 (0x6ba7 bytes) 2024-09-12T09:30:13.993 INFO:teuthology.orchestra.run.smithi007.stdout:1239 write 0xdf5bd04 thru 0xdf668ca (0xabc7 bytes) 2024-09-12T09:30:14.002 INFO:teuthology.orchestra.run.smithi007.stdout:1240 punch from 0x1d0f466 to 0x1d130ae, (0x3c48 bytes) 2024-09-12T09:30:14.002 INFO:teuthology.orchestra.run.smithi007.stdout:1241 write 0x4e0f79d thru 0x4e1d3ee (0xdc52 bytes) 2024-09-12T09:30:14.006 INFO:teuthology.orchestra.run.smithi007.stdout:1242 write 0xd0544fa thru 0xd062b2a (0xe631 bytes) 2024-09-12T09:30:14.015 INFO:teuthology.orchestra.run.smithi007.stdout:1243 read 0xb9e82 thru 0xbb956 (0x1ad5 bytes) 2024-09-12T09:30:14.019 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.019 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:14.019 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:14.019 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:14.023 INFO:teuthology.orchestra.run.smithi007.stdout:1244 trunc from 0xdf668cb to 0x85f154c 2024-09-12T09:30:14.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:1245 punch from 0x510de01 to 0x5116818, (0x8a17 bytes) 2024-09-12T09:30:14.041 INFO:teuthology.orchestra.run.smithi007.stdout:1249 write 0x26ab447 thru 0x26bb0f5 (0xfcaf bytes) 2024-09-12T09:30:14.083 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:14.080+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:1250 punch from 0x7e5ed0a to 0x7e68dad, (0xa0a3 bytes) 2024-09-12T09:30:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:1251 read 0x36a034d thru 0x36ad4f6 (0xd1aa bytes) 2024-09-12T09:30:14.128 INFO:teuthology.orchestra.run.smithi007.stdout:1252 punch from 0x82e4361 to 0x82eda04, (0x96a3 bytes) 2024-09-12T09:30:14.136 INFO:teuthology.orchestra.run.smithi007.stdout:1253 write 0x9511a26 thru 0x951393b (0x1f16 bytes) 2024-09-12T09:30:14.139 INFO:teuthology.orchestra.run.smithi007.stdout:1255 writesame 0x1734ae2 thru 0x1742ada data_size 0x1(0xdff9 bytes) 2024-09-12T09:30:14.152 INFO:teuthology.orchestra.run.smithi007.stdout:1256 trunc from 0x951393c to 0x8263030 2024-09-12T09:30:14.165 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.178 INFO:teuthology.orchestra.run.smithi007.stdout:1258 write 0xdb9ef02 thru 0xdb9f2dd (0x3dc bytes) 2024-09-12T09:30:14.183 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:14.181+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:14.198 INFO:teuthology.orchestra.run.smithi007.stdout:1259 trunc from 0xdb9f2de to 0x82b85b 2024-09-12T09:30:14.206 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:14.206 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:14.206 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:14.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:14.278 INFO:teuthology.orchestra.run.smithi007.stdout:1261 punch from 0x112974 to 0x1190e4, (0x6770 bytes) 2024-09-12T09:30:14.285 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:14.280+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:14.285 INFO:teuthology.orchestra.run.smithi007.stdout:1262 write 0x146710c thru 0x14698f4 (0x27e9 bytes) 2024-09-12T09:30:14.303 INFO:teuthology.orchestra.run.smithi007.stdout:1263 trunc from 0x14698f5 to 0xec430fb 2024-09-12T09:30:14.311 INFO:teuthology.orchestra.run.smithi007.stdout:1264 read 0xe67efbb thru 0xe67f8f1 (0x937 bytes) 2024-09-12T09:30:14.311 INFO:teuthology.orchestra.run.smithi007.stdout:1265 writesame 0x9185916 thru 0x9191784 data_size 0x1(0xbe6f bytes) 2024-09-12T09:30:14.346 INFO:teuthology.orchestra.run.smithi007.stdout:1266 trunc from 0xec430fb to 0x50b9ae5 2024-09-12T09:30:14.356 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.067098471, 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:14.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:14.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.6076022420000005, 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:14.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:14.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58791 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18186240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2342276569999999, 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069510006999999999, 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58791 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121924+0000", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1999999999999998e-07 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121927+0000", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8369999999999999e-06 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121941+0000", 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3733999999999999e-05 2024-09-12T09:30:14.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121952+0000", 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191276+0000", 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069323977999999994 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191337+0000", 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1146e-05 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191433+0000", 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5950000000000001e-05 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.234185291, 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:14.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:14.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:14.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.234138044, 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:14.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2338449680000001, 2024-09-12T09:30:14.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:14.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.233283036, 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2332316729999999, 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2326587929999999, 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:14.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:14.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2325064939999999, 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:14.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2321494040000001, 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:14.405 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:14.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.231524434, 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:14.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2314730979999999, 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:14.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:14.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:14.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2306783079999999, 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:14.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2306034180000001, 2024-09-12T09:30:14.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:14.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2305510980000001, 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:14.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2305025000000001, 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:14.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:14.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.438 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.230248035, 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.439 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:14.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58810 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14483456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:14.443 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2268457699999999, 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069618174000000005, 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58810 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129306+0000", 2024-09-12T09:30:14.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129311+0000", 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129327+0000", 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6446000000000001e-05 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129338+0000", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0865e-05 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198581+0000", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069242608999999997 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198675+0000", 2024-09-12T09:30:14.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4285999999999998e-05 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198923+0000", 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024778499999999999 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.2267937449999999, 2024-09-12T09:30:14.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:14.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:14.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.453 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:14.455 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:14.455 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:14.448+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1268 write 0xedece45 thru 0xedefb69 (0x2d25 bytes) 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1269 read 0x8217b59 thru 0x8220d0d (0x91b5 bytes) 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1270 punch from 0x6cb39a0 to 0x6cbb666, (0x7cc6 bytes) 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1271 read 0x3f56c3f thru 0x3f5f574 (0x8936 bytes) 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1272 read 0x9b5adb2 thru 0x9b5f8bd (0x4b0c bytes) 2024-09-12T09:30:14.456 INFO:teuthology.orchestra.run.smithi007.stdout:1274 read 0x6b7c4bf thru 0x6b840fc (0x7c3e bytes) 2024-09-12T09:30:14.457 INFO:teuthology.orchestra.run.smithi007.stdout:1275 writesame 0xc464fc4 thru 0xc472bdd data_size 0x3(0xdc1a bytes) 2024-09-12T09:30:14.457 INFO:teuthology.orchestra.run.smithi007.stdout:1276 writesame 0x12c6afb thru 0x12d145c data_size 0x63(0xa962 bytes) 2024-09-12T09:30:14.457 INFO:teuthology.orchestra.run.smithi007.stdout:1277 write 0xbd338d5 thru 0xbd3ea71 (0xb19d bytes) 2024-09-12T09:30:14.457 INFO:teuthology.orchestra.run.smithi007.stdout:1278 trunc from 0xedefb6a to 0xc105414 2024-09-12T09:30:14.462 INFO:teuthology.orchestra.run.smithi007.stdout:1279 write 0x6e1e897 thru 0x6e2cb1f (0xe289 bytes) 2024-09-12T09:30:14.504 INFO:teuthology.orchestra.run.smithi007.stdout:1281 trunc from 0xc105414 to 0x52aee 2024-09-12T09:30:14.537 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.555 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:14.552+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:14.579 INFO:teuthology.orchestra.run.smithi007.stdout:1284 write 0xc5fbbe9 thru 0xc604c2f (0x9047 bytes) 2024-09-12T09:30:14.596 INFO:teuthology.orchestra.run.smithi007.stdout:1285 read 0xa4da80a thru 0xa4e6f65 (0xc75c bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1288 read 0x3877ffd thru 0x3881102 (0x9106 bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1289 punch from 0x37e85da to 0x37f4f5c, (0xc982 bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1294 read 0x29e8ed9 thru 0x29e988b (0x9b3 bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1296 punch from 0x56d6a23 to 0x56de648, (0x7c25 bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1298 punch from 0xc5c8dfc to 0xc5d6a61, (0xdc65 bytes) 2024-09-12T09:30:14.597 INFO:teuthology.orchestra.run.smithi007.stdout:1299 write 0x5c77e56 thru 0x5c7f007 (0x71b2 bytes) 2024-09-12T09:30:14.611 INFO:teuthology.orchestra.run.smithi007.stdout:1300 write 0x20edc0b thru 0x20f10e1 (0x34d7 bytes) 2024-09-12T09:30:14.620 INFO:teuthology.orchestra.run.smithi007.stdout:1301 read 0x6336eac thru 0x6339bf3 (0x2d48 bytes) 2024-09-12T09:30:14.628 INFO:teuthology.orchestra.run.smithi007.stdout:1302 trunc from 0xc604c30 to 0x99fd868 2024-09-12T09:30:14.631 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.631 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:14.631 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:14.631 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:14.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:14.651 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.652 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:14.652 INFO:teuthology.orchestra.run.smithi007.stdout:1307 clone 13 order 24 su 65536 sc 15 2024-09-12T09:30:14.692 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:14.692+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:30:14.792 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:14.789+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:14.820 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.820 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:14.820 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:14.820 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:14.820 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:14.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:14.838 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone12 from 0x99fd868 (overlap 0x52aee) to 0x0 2024-09-12T09:30:14.880 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:14.876+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:14.959 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:14.956+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:14.961 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-12T09:30:14.965 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:14.965 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.553369919000001, 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:14.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:14.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.553274656999999, 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:14.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.549590033000001, 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:14.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.547784943, 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:14.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:14.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.547238268000001, 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:14.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:14.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.546469824999999, 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:14.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.546059456999998, 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:14.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:14.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.545920281000001, 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:15.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.538141469999999, 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:15.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:15.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.535777456000002, 2024-09-12T09:30:15.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:15.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:15.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.525025563, 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.524829172, 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.523334752, 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59513 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 20873216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.823237236, 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073624623, 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59513 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140102+0000", 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9400000000000004e-07 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140106+0000", 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9269999999999998e-06 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140118+0000", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140131+0000", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3515e-05 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213318+0000", 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073187167999999997 2024-09-12T09:30:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213409+0000", 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1179000000000002e-05 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213726+0000", 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031658800000000003 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59514 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 20971520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.8231932020000001, 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073929167000000004, 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59514 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140146+0000", 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-07 2024-09-12T09:30:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140150+0000", 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.963e-06 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140162+0000", 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1615e-05 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140172+0000", 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213774+0000", 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073601672000000007 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213848+0000", 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4418999999999993e-05 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214075+0000", 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022613399999999999 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59515 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21069824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.8231223490000001, 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074140521000000001, 2024-09-12T09:30:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59515 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140217+0000", 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140220+0000", 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1860000000000001e-06 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140231+0000", 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0570999999999999e-05 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140237+0000", 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214118+0000", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073880571000000006 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214167+0000", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9271999999999999e-05 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214357+0000", 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018998499999999999 2024-09-12T09:30:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.823091891, 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.8230694729999999, 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.8230463320000001, 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.823021373, 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:15.065 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:15.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:15.067 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:15.061+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:15.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:15.167 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:15.161+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:15.233 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:15.233 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:15.233 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:15.233 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:15.233 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:15.234 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:15.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:15.250 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:15.251 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:15.268 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:15.268 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:15.265+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:15.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:15.285 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:1309 read 0x568b09f thru 0x569acea (0xfc4c bytes) 2024-09-12T09:30:15.323 INFO:teuthology.orchestra.run.smithi007.stdout:1312 write 0x47f82b thru 0x4829a5 (0x317b bytes) 2024-09-12T09:30:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:1315 read 0x701aceb thru 0x7028868 (0xdb7e bytes) 2024-09-12T09:30:15.369 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:15.365+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:1316 trunc from 0x99fd868 to 0x44425c7 2024-09-12T09:30:15.377 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:15.377 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:15.378 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:15.378 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:15.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:15.470 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:15.468+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:15.551 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:15.555 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:15.558 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:15.558 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:15.570 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:15.568+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:15.571 INFO:teuthology.orchestra.run.smithi007.stdout:1318 write 0xe728b9e thru 0xe72c99a (0x3dfd bytes) 2024-09-12T09:30:15.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:15.574 INFO:teuthology.orchestra.run.smithi007.stdout:1319 punch from 0xa26f190 to 0xa276f35, (0x7da5 bytes) 2024-09-12T09:30:15.574 INFO:teuthology.orchestra.run.smithi007.stdout:1320 writesame 0x34280aa thru 0x3432cbb data_size 0xa(0xac12 bytes) 2024-09-12T09:30:15.574 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:15.575 INFO:teuthology.orchestra.run.smithi007.stdout:1321 write 0x7ae2c81 thru 0x7ae6dc7 (0x4147 bytes) 2024-09-12T09:30:15.577 INFO:teuthology.orchestra.run.smithi007.stdout:1322 writesame 0x3fa317a thru 0x3fab765 data_size 0x6(0x85ec bytes) 2024-09-12T09:30:15.581 INFO:teuthology.orchestra.run.smithi007.stdout:1323 read 0x5d1a540 thru 0x5d25d23 (0xb7e4 bytes) 2024-09-12T09:30:15.581 INFO:teuthology.orchestra.run.smithi007.stdout:1324 read 0x1b724b thru 0x1bef78 (0x7d2e bytes) 2024-09-12T09:30:15.582 INFO:teuthology.orchestra.run.smithi007.stdout:1326 punch from 0xad3d508 to 0xad40036, (0x2b2e bytes) 2024-09-12T09:30:15.582 INFO:teuthology.orchestra.run.smithi007.stdout:1327 clone 14 order 21 su 8192 sc 9 2024-09-12T09:30:15.671 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:15.668+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:15.731 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:15.731 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:15.748 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:15.771 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:15.765+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:15.872 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:15.869+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:15.888 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone13 from 0xe72c99b (overlap 0x44425c7) to 0x22212e3 2024-09-12T09:30:15.929 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:15.929 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:15.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:15.991 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:15.992+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:16.065 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:16.066 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:16.074 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:16.072+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:16.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:16.088 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-12T09:30:16.092 INFO:teuthology.orchestra.run.smithi007.stdout:1328 writesame 0x102e688 thru 0x103d243 data_size 0x5e(0xebbc bytes) 2024-09-12T09:30:16.175 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:16.169+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:16.251 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:16.251 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:16.266 INFO:teuthology.orchestra.run.smithi007.stdout:1329 trunc from 0xe72c99b to 0xe210f80 2024-09-12T09:30:16.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:16.312 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:16.309+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:16.353 INFO:teuthology.orchestra.run.smithi007.stdout:1330 read 0xb981284 thru 0xb989744 (0x84c1 bytes) 2024-09-12T09:30:16.354 INFO:teuthology.orchestra.run.smithi007.stdout:1331 read 0xe07e4b7 thru 0xe08480e (0x6358 bytes) 2024-09-12T09:30:16.355 INFO:teuthology.orchestra.run.smithi007.stdout:1332 punch from 0xa8d5798 to 0xa8dda8b, (0x82f3 bytes) 2024-09-12T09:30:16.355 INFO:teuthology.orchestra.run.smithi007.stdout:1333 write 0xbee5fa6 thru 0xbeef959 (0x99b4 bytes) 2024-09-12T09:30:16.376 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:16.373+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:16.441 INFO:teuthology.orchestra.run.smithi007.stdout:1334 writesame 0x7089a31 thru 0x708d0e9 data_size 0x1(0x36b9 bytes) 2024-09-12T09:30:16.477 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:16.476+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:16.477 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:16.478 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:16.487 INFO:teuthology.orchestra.run.smithi007.stdout:1335 trunc from 0xe210f80 to 0xc42d297 2024-09-12T09:30:16.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:16.546 INFO:teuthology.orchestra.run.smithi007.stdout:1336 punch from 0x8c77443 to 0x8c7cc26, (0x57e3 bytes) 2024-09-12T09:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout:1342 writesame 0xc84ea19 thru 0xc85b059 data_size 0x1(0xc641 bytes) 2024-09-12T09:30:16.553 INFO:teuthology.orchestra.run.smithi007.stdout:1343 read 0x3b70f30 thru 0x3b71c70 (0xd41 bytes) 2024-09-12T09:30:16.554 INFO:teuthology.orchestra.run.smithi007.stdout:1344 read 0x4eb33a0 thru 0x4ebe711 (0xb372 bytes) 2024-09-12T09:30:16.555 INFO:teuthology.orchestra.run.smithi007.stdout:1346 punch from 0x31a1ce6 to 0x31a9262, (0x757c bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1347 punch from 0x7b53be3 to 0x7b5e921, (0xad3e bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1349 punch from 0x3c50d70 to 0x3c52d64, (0x1ff4 bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1351 punch from 0x5fd4742 to 0x5fd4d46, (0x604 bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1355 punch from 0x586a550 to 0x587933d, (0xeded bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1356 read 0x3c0614 thru 0x3c9d99 (0x9786 bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1357 read 0xaeeb6a9 thru 0xaeebe5c (0x7b4 bytes) 2024-09-12T09:30:16.556 INFO:teuthology.orchestra.run.smithi007.stdout:1359 write 0x7225fef thru 0x722883c (0x284e bytes) 2024-09-12T09:30:16.572 INFO:teuthology.orchestra.run.smithi007.stdout:1360 punch from 0x5b89c16 to 0x5b89dd6, (0x1c0 bytes) 2024-09-12T09:30:16.572 INFO:teuthology.orchestra.run.smithi007.stdout:1361 writesame 0xbca142c thru 0xbca15ec data_size 0x1(0x1c1 bytes) 2024-09-12T09:30:16.578 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:16.576+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:16.583 INFO:teuthology.orchestra.run.smithi007.stdout:1362 write 0xe44cae8 thru 0xe453137 (0x6650 bytes) 2024-09-12T09:30:16.589 INFO:teuthology.orchestra.run.smithi007.stdout:1363 read 0xa2becbd thru 0xa2c5fb9 (0x72fd bytes) 2024-09-12T09:30:16.589 INFO:teuthology.orchestra.run.smithi007.stdout:1364 write 0x7076f1c thru 0x7078abd (0x1ba2 bytes) 2024-09-12T09:30:16.591 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.591 INFO:teuthology.orchestra.run.smithi007.stdout:1368 read 0x1a97b05 thru 0x1aa1399 (0x9895 bytes) 2024-09-12T09:30:16.592 INFO:teuthology.orchestra.run.smithi007.stdout:1369 read 0x836b5ba thru 0x836dbb3 (0x25fa bytes) 2024-09-12T09:30:16.592 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.593 INFO:teuthology.orchestra.run.smithi007.stdout:1371 write 0x270a3b5 thru 0x270ebef (0x483b bytes) 2024-09-12T09:30:16.642 INFO:teuthology.orchestra.run.smithi007.stdout:1372 punch from 0xb7485da to 0xb7565f2, (0xe018 bytes) 2024-09-12T09:30:16.643 INFO:teuthology.orchestra.run.smithi007.stdout:1373 write 0xbbc7848 thru 0xbbd1158 (0x9911 bytes) 2024-09-12T09:30:16.645 INFO:teuthology.orchestra.run.smithi007.stdout:1375 write 0x455712c thru 0x45624eb (0xb3c0 bytes) 2024-09-12T09:30:16.679 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:16.676+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:16.706 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:16.706 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:16.724 INFO:teuthology.orchestra.run.smithi007.stdout:1376 punch from 0x4e118a1 to 0x4e17998, (0x60f7 bytes) 2024-09-12T09:30:16.725 INFO:teuthology.orchestra.run.smithi007.stdout:1377 write 0x629c4d7 thru 0x62aa3a2 (0xdecc bytes) 2024-09-12T09:30:16.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:16.736 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [6, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:16.736 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:16.736 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-12T09:30:16.736 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T09:30:16.736 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T09:30:16.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-12T09:30:16.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:30:16.780 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:16.780+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:16.812 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.812 INFO:teuthology.orchestra.run.smithi007.stdout:1380 write 0x62b14cc thru 0x62baa6d (0x95a2 bytes) 2024-09-12T09:30:16.814 INFO:teuthology.orchestra.run.smithi007.stdout:1381 write 0x373b1ad thru 0x374a91f (0xf773 bytes) 2024-09-12T09:30:16.853 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.853 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:16.853 INFO:teuthology.orchestra.run.smithi007.stdout:1384 writesame 0xce77f9c thru 0xce7c652 data_size 0x1(0x46b7 bytes) 2024-09-12T09:30:16.862 INFO:teuthology.orchestra.run.smithi007.stdout:1385 writesame 0x35f56ee thru 0x35fc85a data_size 0x1(0x716d bytes) 2024-09-12T09:30:16.881 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:16.880+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:16.962 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T09:30:16.962 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:16.964 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:16.964 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:16.970 INFO:teuthology.orchestra.run.smithi007.stdout:1386 read 0x7738e6a thru 0x7741c3f (0x8dd6 bytes) 2024-09-12T09:30:16.971 INFO:teuthology.orchestra.run.smithi007.stdout:1387 write 0x7d5c2f3 thru 0x7d61e10 (0x5b1e bytes) 2024-09-12T09:30:16.982 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:16.980+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:16.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:17.038 INFO:teuthology.orchestra.run.smithi007.stdout:1388 punch from 0xa959580 to 0xa95a1e3, (0xc63 bytes) 2024-09-12T09:30:17.038 INFO:teuthology.orchestra.run.smithi007.stdout:1389 clone 15 order 23 su 131072 sc 11 2024-09-12T09:30:17.083 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:17.076+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:17.140 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:17.141 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:17.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:17.184 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:17.184+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:17.285 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:17.284+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:17.306 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:17.306 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:17.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:17.386 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:17.380+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:17.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:17.487 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:17.484+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:17.588 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:17.588+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:17.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:17.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:17.728 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:17.724+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:17.741 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:30:17.761 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:17.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:17.797 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:17.798 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:17.792+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:17.810 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1062 3.9 e28/23 3:9649ecbc:::rbd_data.1191192d73a0.0000000000000017:head v 28'6, mlcod=28'4)", 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.275056640999999, 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024265419999999998, 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534712+0000", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534714+0000", 2024-09-12T09:30:17.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.88e-06 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534753+0000", 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9212999999999999e-05 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534777+0000", 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4533e-05 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534790+0000", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2577e-05 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534919+0000", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956600000000001 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.534980+0000", 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0294999999999998e-05 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537034+0000", 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020547149999999999 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.537138+0000", 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103763 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:1131 3.0 e28/23 3:02e1fe0e:::rbd_data.1191192d73a0.0000000000000029:head v 28'14, mlcod=28'12)", 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.097395229, 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024551260000000002, 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712373+0000", 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712376+0000", 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712437+0000", 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1660000000000003e-05 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712473+0000", 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6103000000000003e-05 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712495+0000", 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1982999999999999e-05 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712671+0000", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017596800000000001 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.712713+0000", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1334000000000001e-05 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714742+0000", 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002029037 2024-09-12T09:30:17.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:40.714828+0000", 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6546999999999997e-05 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.678623382000001, 2024-09-12T09:30:17.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032699309999999998, 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131145+0000", 2024-09-12T09:30:17.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.131146+0000", 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132482+0000", 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013359159999999999 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132558+0000", 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5488000000000006e-05 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132574+0000", 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132703+0000", 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012893299999999999 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.132722+0000", 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8821000000000002e-05 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.134415+0000", 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016933460000000001 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "MOSDPGPush(3.1f 38/37 [PushOp(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5, version: 33'271, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:f8dc5c6a:::rbd_data.1191d634a7ff.0000000000000001:5@33'271, size: 16453796, copy_subset: [(0, 16453796)], clone_subset: {}, snapset: 5=[]:{5: [5]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-12T09:30:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.671814704999999, 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024217790000000002, 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137954+0000", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.137955+0000", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.499e-06 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.138990+0000", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010348410000000001 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139025+0000", 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5367999999999997e-05 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139045+0000", 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9191e-05 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139179+0000", 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134728 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.139198+0000", 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8899e-05 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:50.140375+0000", 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011772530000000001 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:11692 3.4 e38/23 3:241fb2a9:::rbd_data.11913fee8ce1.0000000000000028:head v 38'8, mlcod=38'6)", 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.773877312, 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023808789999999998, 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035891+0000", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035893+0000", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035924+0000", 2024-09-12T09:30:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1569999999999998e-05 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035945+0000", 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0392000000000001e-05 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.035954+0000", 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.516e-06 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036068+0000", 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011424199999999999 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.036105+0000", 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6871e-05 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038166+0000", 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002060236 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:51.038272+0000", 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106331 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5103543699999999, 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:30:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.3279354520000002, 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:30:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54570 3.9 e55/23 3:9606e935:::rbd_data.1191175d8602.000000000000000c:head v 55'55, mlcod=49'52)", 2024-09-12T09:30:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.3166378329999997, 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037370770000000001, 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493131+0000", 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493133+0000", 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3329999999999999e-06 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493160+0000", 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6934e-05 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493178+0000", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7972e-05 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.493191+0000", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2773e-05 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.495621+0000", 2024-09-12T09:30:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024305020000000002 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.495649+0000", 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8350000000000001e-05 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496822+0000", 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011723059999999999 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496868+0000", 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5906999999999999e-05 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:64830 3.0 e57/23 3:036ce656:::rbd_data.11918dbccc64.0000000000000002:head v 57'75, mlcod=57'73)", 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.867162+0000", 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.9426060659999997, 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034291679999999998, 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867162+0000", 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867162+0000", 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867164+0000", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.271e-06 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867890+0000", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072610899999999996 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867914+0000", 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4405000000000001e-05 2024-09-12T09:30:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.867924+0000", 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0009e-05 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.868091+0000", 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016653300000000001 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.868113+0000", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2120000000000002e-05 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.870541+0000", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024282269999999998 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.870591+0000", 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0494000000000001e-05 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:64832 3.1e e57/46 3:7e880a99:::rbd_data.11918dbccc64.0000000000000007:head v 57'68, mlcod=57'66)", 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.870971+0000", 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.9387974780000001, 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026855590000000001, 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.870971+0000", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.870971+0000", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.870972+0000", 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999998e-07 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.871467+0000", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049534299999999998 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.871486+0000", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8906000000000002e-05 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.871496+0000", 2024-09-12T09:30:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0158e-05 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.871559+0000", 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3003999999999995e-05 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.871584+0000", 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4893999999999999e-05 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.873608+0000", 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002023417 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.873656+0000", 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8893000000000001e-05 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66703 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'349, mlcod=57'347)", 2024-09-12T09:30:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.399315+0000", 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4104537220000002, 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024003710000000001, 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399315+0000", 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399315+0000", 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399316+0000", 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6700000000000001e-06 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399327+0000", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389e-05 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399350+0000", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3363e-05 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399362+0000", 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1827e-05 2024-09-12T09:30:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399472+0000", 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110287 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.399508+0000", 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5899000000000003e-05 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401597+0000", 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020894220000000001 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401715+0000", 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117514 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66705 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'351, mlcod=57'347)", 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.401319+0000", 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4084490820000002, 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027211739999999998, 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401319+0000", 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401319+0000", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401321+0000", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4470000000000001e-06 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401330+0000", 2024-09-12T09:30:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9930000000000001e-06 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401353+0000", 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3159999999999998e-05 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401397+0000", 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3955000000000002e-05 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401773+0000", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000376404 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.401816+0000", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2722000000000002e-05 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.403927+0000", 2024-09-12T09:30:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021115359999999998 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404040+0000", 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112957 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66706 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'352, mlcod=57'347)", 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.402975+0000", 2024-09-12T09:30:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4067930049999999, 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00269148, 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.402975+0000", 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.402975+0000", 2024-09-12T09:30:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.402977+0000", 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.33e-06 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.402984+0000", 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4599999999999997e-06 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.403007+0000", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3119e-05 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.403030+0000", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3232999999999999e-05 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.403162+0000", 2024-09-12T09:30:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131326 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.403197+0000", 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5369e-05 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405528+0000", 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023309530000000002 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405667+0000", 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013868999999999999 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66707 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'353, mlcod=57'347)", 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.404330+0000", 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4054381149999999, 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027003470000000001, 2024-09-12T09:30:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404330+0000", 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404330+0000", 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404332+0000", 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.39e-06 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404341+0000", 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9330000000000007e-06 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404362+0000", 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1282999999999999e-05 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404373+0000", 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1353e-05 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404537+0000", 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164179 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.404571+0000", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3936000000000002e-05 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.406834+0000", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002262554 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.407031+0000", 2024-09-12T09:30:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019671900000000001 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66708 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'354, mlcod=57'350)", 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.405593+0000", 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4041749490000002, 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027143290000000001, 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405593+0000", 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405593+0000", 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405595+0000", 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.68e-06 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405605+0000", 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0076e-05 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405661+0000", 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6298000000000002e-05 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405687+0000", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5752999999999999e-05 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405830+0000", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014285 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.405884+0000", 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3752000000000003e-05 2024-09-12T09:30:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.408264+0000", 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023805020000000001 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.408308+0000", 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3417999999999997e-05 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66774 3.9 e57/23 3:959c1016:::rbd_data.11918dbccc64.0000000000000037:head v 57'83, mlcod=57'81)", 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.512033+0000", 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.2977354440000002, 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026107190000000001, 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512033+0000", 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512033+0000", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512034+0000", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5600000000000001e-06 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512068+0000", 2024-09-12T09:30:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3609999999999998e-05 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512092+0000", 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3427000000000001e-05 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512104+0000", 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2897999999999999e-05 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512304+0000", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019952500000000001 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.512361+0000", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7321000000000002e-05 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.514518+0000", 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021569710000000001 2024-09-12T09:30:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.514644+0000", 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012540699999999999 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66837 3.14 e57/54 3:2e2b2254:::rbd_object_map.11918dbccc64:head v 57'383, mlcod=57'380)", 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.604698+0000", 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.205070573, 2024-09-12T09:30:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026539150000000002, 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604698+0000", 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604698+0000", 2024-09-12T09:30:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604699+0000", 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.547e-06 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604709+0000", 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9259999999999995e-06 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604732+0000", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3246000000000001e-05 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604749+0000", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.645e-05 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604852+0000", 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103371 2024-09-12T09:30:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.604890+0000", 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7685000000000003e-05 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.607215+0000", 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023246429999999999 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.607352+0000", 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137047 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66961 3.1e e58/46 3:7e880a99:::rbd_data.11918dbccc64.0000000000000007:head v 58'88, mlcod=57'85)", 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.846554+0000", 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.963214641, 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0041342790000000003, 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846554+0000", 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846554+0000", 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846555+0000", 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1399999999999995e-07 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846565+0000", 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0312999999999999e-05 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846605+0000", 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0231999999999997e-05 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846627+0000", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1393e-05 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846780+0000", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015381300000000001 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846837+0000", 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6984999999999997e-05 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.850598+0000", 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037603530000000001 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.850688+0000", 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0276000000000001e-05 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66956 3.0 e58/23 3:036ce656:::rbd_data.11918dbccc64.0000000000000002:head v 58'86, mlcod=57'83)", 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.846663+0000", 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.9631056440000001, 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028347060000000002, 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846663+0000", 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846663+0000", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846664+0000", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0999999999999997e-07 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846673+0000", 2024-09-12T09:30:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9040000000000004e-06 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846699+0000", 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6061999999999999e-05 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846716+0000", 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6606999999999999e-05 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846871+0000", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000155289 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.846925+0000", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3365999999999997e-05 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.849431+0000", 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025062560000000001 2024-09-12T09:30:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.849497+0000", 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6412e-05 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:66965 3.13 e58/51 3:c9aa216b:::rbd_data.11918dbccc64.000000000000000c:head v 58'60, mlcod=57'57)", 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:14.853701+0000", 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.9560675720000003, 2024-09-12T09:30:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029999319999999999, 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853701+0000", 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853701+0000", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853702+0000", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.621e-06 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853716+0000", 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3242e-05 2024-09-12T09:30:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853746+0000", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0471999999999999e-05 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853762+0000", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6339000000000001e-05 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.853913+0000", 2024-09-12T09:30:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015046100000000001 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.854005+0000", 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2276999999999998e-05 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.856600+0000", 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025949200000000001 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:14.856701+0000", 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010060000000000001 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:17.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:17.905 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:17.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:17.907 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:17.908 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone14 intact 2024-09-12T09:30:17.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:17.909 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:17.904+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:18.006 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:18.000+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:18.066 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:18.066 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:18.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:18.107 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:18.104+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:18.112 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-12T09:30:18.174 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.187 INFO:teuthology.orchestra.run.smithi007.stdout:1391 trunc from 0xe453138 to 0x2d116b4 2024-09-12T09:30:18.209 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:18.208+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:18.270 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:18.271 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:18.271 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:18.271 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:18.271 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:18.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:18.344 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:18.336+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:18.410 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:18.408+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:18.497 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53986 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 14909440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1084410640000009, 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055388721000000002, 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53986 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387069+0000", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.905e-06 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387076+0000", 2024-09-12T09:30:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387087+0000", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1499e-05 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442148+0000", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055060410999999997 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442229+0000", 2024-09-12T09:30:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1441999999999997e-05 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442456+0000", 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000226534 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53987 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15007744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1084126609999991, 2024-09-12T09:30:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055709171000000002, 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53987 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387096+0000", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387098+0000", 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0770000000000002e-06 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387105+0000", 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7619999999999998e-06 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387113+0000", 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4959999999999996e-06 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442489+0000", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055376140999999997 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442610+0000", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120234 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442805+0000", 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019502899999999999 2024-09-12T09:30:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53988 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15106048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1083872689999996, 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057056229, 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53988 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387123+0000", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387130+0000", 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5810000000000002e-06 2024-09-12T09:30:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387135+0000", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1490000000000002e-06 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442836+0000", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055700889000000003 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442918+0000", 2024-09-12T09:30:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000004e-05 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.444177+0000", 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012590570000000001 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54006 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17367040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:18.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1081808510000002, 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056089721000000002, 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54006 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389586+0000", 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002258552 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389632+0000", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5796e-05 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389644+0000", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2699e-05 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389667+0000", 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3076999999999999e-05 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443194+0000", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053526642999999999 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443320+0000", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012563599999999999 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443417+0000", 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7318000000000006e-05 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54034 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17694720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1044180860000008, 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055444460000000001, 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54034 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391755+0000", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066475499999999995 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391760+0000", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.181e-06 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391904+0000", 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014413300000000001 2024-09-12T09:30:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391916+0000", 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1507e-05 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446235+0000", 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054319522000000002 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446313+0000", 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7745999999999997e-05 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446534+0000", 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221616 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54051 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20250624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1027470990000001, 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055664077999999999, 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54051 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392762+0000", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392764+0000", 2024-09-12T09:30:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4710000000000001e-06 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392772+0000", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6969999999999993e-06 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392777+0000", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.587e-06 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447587+0000", 2024-09-12T09:30:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.05480952 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447661+0000", 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4635000000000006e-05 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448425+0000", 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076352799999999997 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54052 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20348928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1027203780000008, 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056119461000000002, 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54052 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393045+0000", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025747600000000003 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393054+0000", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0159999999999997e-06 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393064+0000", 2024-09-12T09:30:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4050000000000006e-06 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393071+0000", 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6290000000000001e-06 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448481+0000", 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055409753999999999 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448576+0000", 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4969999999999994e-05 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448907+0000", 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033121099999999998 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61223 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13336576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2559266530000004, 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055340981999999997, 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61223 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239582+0000", 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3500000000000002e-07 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239584+0000", 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3860000000000001e-06 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239635+0000", 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0985000000000001e-05 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239643+0000", 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8869999999999992e-06 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294604+0000", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054961083000000001 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294717+0000", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011312 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294922+0000", 2024-09-12T09:30:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020508599999999999 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61227 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13533184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2552545799999999, 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055556323999999997, 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61227 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240254+0000", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240257+0000", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.762e-06 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240265+0000", 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8469999999999996e-06 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240271+0000", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0689999999999998e-06 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295548+0000", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055277706000000003 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295630+0000", 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1450000000000001e-05 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295810+0000", 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017996399999999999 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61229 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13631488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.255228003, 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055868963000000001, 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61229 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240281+0000", 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240283+0000", 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240290+0000", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0570000000000004e-06 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240295+0000", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.164e-06 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295854+0000", 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055558626999999999 2024-09-12T09:30:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295952+0000", 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8405000000000004e-05 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296149+0000", 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019671099999999999 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61231 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13729792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2552039019999999, 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056138067999999999, 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61231 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240305+0000", 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240307+0000", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.294e-06 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240315+0000", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1019999999999997e-06 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240321+0000", 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9370000000000001e-06 2024-09-12T09:30:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296180+0000", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055859345999999997 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296248+0000", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7596999999999997e-05 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296442+0000", 2024-09-12T09:30:18.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019433699999999999 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61233 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13828096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2551791369999998, 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056534344, 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61233 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240332+0000", 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240340+0000", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240346+0000", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296492+0000", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056146433000000003 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296606+0000", 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011335799999999999 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296863+0000", 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000257612 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61235 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13926400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:18.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2551526219999998, 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056785587999999998, 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61235 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:18.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240356+0000", 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240358+0000", 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5390000000000002e-06 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240367+0000", 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6750000000000008e-06 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240372+0000", 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2789999999999998e-06 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296882+0000", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056509707999999999 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296969+0000", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7366000000000003e-05 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297141+0000", 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017161100000000001 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61237 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14024704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2551281699999999, 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057667507, 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61237 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9400000000000001e-07 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240382+0000", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1900000000000002e-06 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240389+0000", 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4060000000000002e-06 2024-09-12T09:30:18.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240394+0000", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9300000000000002e-06 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297172+0000", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056777785999999997 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297313+0000", 2024-09-12T09:30:18.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014155099999999999 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298047+0000", 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073424999999999999 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61239 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:18.560 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2551060280000002, 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057981773, 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61239 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9000000000000002e-07 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240405+0000", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240412+0000", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.469e-06 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240427+0000", 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5102e-05 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298080+0000", 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057653049999999997 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298166+0000", 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5296000000000005e-05 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298384+0000", 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000218027 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61241 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14221312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2550700270000004, 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058244112000000001, 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61241 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:18.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240441+0000", 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7020000000000001e-06 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240450+0000", 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240455+0000", 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298403+0000", 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057947945000000001 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298473+0000", 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9863999999999994e-05 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298682+0000", 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020926199999999999 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61243 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14319616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2550450250000003, 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058591126, 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61243 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240466+0000", 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7010000000000001e-06 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240473+0000", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240479+0000", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.327e-06 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298722+0000", 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058243504000000001 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298783+0000", 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1137999999999996e-05 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299054+0000", 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027089799999999999 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61245 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14417920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2550212519999997, 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058955593000000001, 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61245 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240494+0000", 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5420000000000002e-06 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240511+0000", 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240518+0000", 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5989999999999998e-06 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299110+0000", 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058592712999999998 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299190+0000", 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9622000000000004e-05 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299442+0000", 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025237399999999998 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61247 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14516224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2549805779999996, 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059307199999999997, 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61247 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240528+0000", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240531+0000", 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.615e-06 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240539+0000", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240545+0000", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0310000000000004e-06 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299503+0000", 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058957756 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299625+0000", 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012207099999999999 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299835+0000", 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209605 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61249 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14614528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.2549542919999999, 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059607902999999997, 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61249 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:18.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-12T09:30:18.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240557+0000", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5780000000000001e-06 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240565+0000", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5040000000000002e-06 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240570+0000", 2024-09-12T09:30:18.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0690000000000001e-06 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299884+0000", 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059313728000000003 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299967+0000", 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3404000000000003e-05 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.300162+0000", 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019417999999999999 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:18.589 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:18.590 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:18.591 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:18.588+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:18.672 INFO:teuthology.orchestra.run.smithi007.stdout:1392 write 0x9babb4e thru 0x9bb4a10 (0x8ec3 bytes) 2024-09-12T09:30:18.676 INFO:teuthology.orchestra.run.smithi007.stdout:1393 punch from 0x2c001bf to 0x2c055a5, (0x53e6 bytes) 2024-09-12T09:30:18.677 INFO:teuthology.orchestra.run.smithi007.stdout:1394 write 0x1cca87 thru 0x1d804b (0xb5c5 bytes) 2024-09-12T09:30:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:1395 write 0xbfb629d thru 0xbfb9090 (0x2df4 bytes) 2024-09-12T09:30:18.691 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:18.688+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:18.702 INFO:teuthology.orchestra.run.smithi007.stdout:1396 writesame 0xdc5f353 thru 0xdc6d6b8 data_size 0xd(0xe366 bytes) 2024-09-12T09:30:18.702 INFO:teuthology.orchestra.run.smithi007.stdout:1397 writesame 0x9fe245c thru 0x9fe9dd7 data_size 0x32(0x797c bytes) 2024-09-12T09:30:18.720 INFO:teuthology.orchestra.run.smithi007.stdout:1398 trunc from 0xdc6d6b9 to 0x21ffb06 2024-09-12T09:30:18.723 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:95441 3.14 3:28a10ebd:::rbd_object_map.11917e1da991:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:18.722261+0000", 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021738830000000002, 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022190180000000001, 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:18.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 95441 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722261+0000", 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722261+0000", 2024-09-12T09:30:18.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722263+0000", 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3439999999999999e-06 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722275+0000", 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722307+0000", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2104999999999998e-05 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722333+0000", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6329000000000002e-05 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722429+0000", 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6168999999999994e-05 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722535+0000", 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105389 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.722913+0000", 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037774599999999998 2024-09-12T09:30:18.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:18.724119+0000", 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012064319999999999 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:30:18.729 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:18.739 INFO:teuthology.orchestra.run.smithi007.stdout:1399 write 0x1351b56 thru 0x1357efc (0x63a7 bytes) 2024-09-12T09:30:18.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:18.741 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.741 INFO:teuthology.orchestra.run.smithi007.stdout:1402 read 0x11628f6 thru 0x117103d (0xe748 bytes) 2024-09-12T09:30:18.742 INFO:teuthology.orchestra.run.smithi007.stdout:1403 read 0x177d030 thru 0x1789cb8 (0xcc89 bytes) 2024-09-12T09:30:18.742 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.762 INFO:teuthology.orchestra.run.smithi007.stdout:1405 write 0x93797ac thru 0x937cc8d (0x34e2 bytes) 2024-09-12T09:30:18.767 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.767 INFO:teuthology.orchestra.run.smithi007.stdout:1407 read 0x1140372 thru 0x114e483 (0xe112 bytes) 2024-09-12T09:30:18.768 INFO:teuthology.orchestra.run.smithi007.stdout:1410 read 0x69303e6 thru 0x693d527 (0xd142 bytes) 2024-09-12T09:30:18.768 INFO:teuthology.orchestra.run.smithi007.stdout:1411 punch from 0x9183a54 to 0x9190253, (0xc7ff bytes) 2024-09-12T09:30:18.774 INFO:teuthology.orchestra.run.smithi007.stdout:1412 trunc from 0x937cc8e to 0xd3f12e6 2024-09-12T09:30:18.783 INFO:teuthology.orchestra.run.smithi007.stdout:1414 punch from 0x553ce8b to 0x55400b3, (0x3228 bytes) 2024-09-12T09:30:18.783 INFO:teuthology.orchestra.run.smithi007.stdout:1415 writesame 0x49e9363 thru 0x49e97f1 data_size 0x1(0x48f bytes) 2024-09-12T09:30:18.785 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.785 INFO:teuthology.orchestra.run.smithi007.stdout:1417 writesame 0x993a034 thru 0x99457a1 data_size 0x35(0xb76e bytes) 2024-09-12T09:30:18.792 INFO:teuthology.orchestra.run.smithi007.stdout:1418 trunc from 0xd3f12e6 to 0xd7d8eed 2024-09-12T09:30:18.799 INFO:teuthology.orchestra.run.smithi007.stdout:1419 write 0x3172a48 thru 0x317580f (0x2dc8 bytes) 2024-09-12T09:30:18.802 INFO:teuthology.orchestra.run.smithi007.stdout:1420 write 0x869102b thru 0x869d45d (0xc433 bytes) 2024-09-12T09:30:18.807 INFO:teuthology.orchestra.run.smithi007.stdout:1422 write 0x86d3556 thru 0x86d4366 (0xe11 bytes) 2024-09-12T09:30:18.812 INFO:teuthology.orchestra.run.smithi007.stdout:1423 write 0x324c670 thru 0x325545e (0x8def bytes) 2024-09-12T09:30:18.814 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.814 INFO:teuthology.orchestra.run.smithi007.stdout:1425 write 0xb4c140d thru 0xb4c6d3d (0x5931 bytes) 2024-09-12T09:30:18.823 INFO:teuthology.orchestra.run.smithi007.stdout:1426 trunc from 0xd7d8eed to 0xe9997d0 2024-09-12T09:30:18.831 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.832 INFO:teuthology.orchestra.run.smithi007.stdout:1428 writesame 0xc64f8cc thru 0xc6560ce data_size 0x1(0x6803 bytes) 2024-09-12T09:30:18.832 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:18.832+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:18.862 INFO:teuthology.orchestra.run.smithi007.stdout:1430 read 0xd15abb3 thru 0xd1689e0 (0xde2e bytes) 2024-09-12T09:30:18.863 INFO:teuthology.orchestra.run.smithi007.stdout:1431 writesame 0x9439ee2 thru 0x9446b0c data_size 0x1(0xcc2b bytes) 2024-09-12T09:30:18.893 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:18.892+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:18.897 INFO:teuthology.orchestra.run.smithi007.stdout:1432 write 0xcd9a5d0 thru 0xcda84a1 (0xded2 bytes) 2024-09-12T09:30:18.904 INFO:teuthology.orchestra.run.smithi007.stdout:1433 trunc from 0xe9997d0 to 0xe557018 2024-09-12T09:30:18.912 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:18.912 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:18.913 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:18.913 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:18.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:18.933 INFO:teuthology.orchestra.run.smithi007.stdout:1435 write 0xed3fd21 thru 0xed3ff1c (0x1fc bytes) 2024-09-12T09:30:18.936 INFO:teuthology.orchestra.run.smithi007.stdout:1436 read 0x1ac1e43 thru 0x1ac7cfe (0x5ebc bytes) 2024-09-12T09:30:18.936 INFO:teuthology.orchestra.run.smithi007.stdout:1438 writesame 0xd6b1861 thru 0xd6b1b21 data_size 0x5(0x2c1 bytes) 2024-09-12T09:30:18.943 INFO:teuthology.orchestra.run.smithi007.stdout:1439 clone 16 order 22 su 32768 sc 12 2024-09-12T09:30:18.994 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:18.988+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:19.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.802053904999999, 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:19.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.342557676, 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:19.098 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:19.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58791 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18186240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.103 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9691830909999997, 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069510006999999999, 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58791 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121923+0000", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121924+0000", 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1999999999999998e-07 2024-09-12T09:30:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121927+0000", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8369999999999999e-06 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121941+0000", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3733999999999999e-05 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121952+0000", 2024-09-12T09:30:19.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191276+0000", 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069323977999999994 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191337+0000", 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1146e-05 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191433+0000", 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5950000000000001e-05 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9691407249999999, 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.108 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:19.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:19.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9690934779999996, 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.113 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:19.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9688004020000003, 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:19.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9682384700000002, 2024-09-12T09:30:19.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:19.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9681871070000003, 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.127 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:19.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9676142270000003, 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:19.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:19.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9674619279999996, 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:19.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.967104838, 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:19.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9664798680000004, 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:19.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:19.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:19.150 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9664285320000001, 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:19.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:19.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9656337419999996, 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:19.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.965558852, 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:19.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:19.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.965506532, 2024-09-12T09:30:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9654579339999998, 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9652034690000004, 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58810 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14483456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9618012040000004, 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069618174000000005, 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58810 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129306+0000", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129311+0000", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129327+0000", 2024-09-12T09:30:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6446000000000001e-05 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129338+0000", 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0865e-05 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198581+0000", 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069242608999999997 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198675+0000", 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4285999999999998e-05 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198923+0000", 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024778499999999999 2024-09-12T09:30:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9617491789999999, 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.189 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:19.190 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:19.190 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:19.188+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:19.290 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:19.288+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:19.363 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:19.363 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:19.363 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:19.363 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:19.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:19.391 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:19.384+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:19.492 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:19.488+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:19.541 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:19.541 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:19.541 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:19.541 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:19.541 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:19.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:19.593 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:19.588+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:19.679 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:19.679 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:19.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.267490338999998, 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:19.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.267395077, 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:19.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:19.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.263710453000002, 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:19.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:19.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.261905363, 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:19.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:19.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.261358688000001, 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:19.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:19.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.260590244999999, 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:19.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:19.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.260179876999999, 2024-09-12T09:30:19.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:19.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:19.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.260040701000001, 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:19.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.25226189, 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:19.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:19.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.249897875999999, 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:19.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:19.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.239145983, 2024-09-12T09:30:19.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:19.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:19.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:19.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.238949592000001, 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:19.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:19.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 30.237455172000001, 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:19.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:19.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59513 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 20873216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5373576560000002, 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073624623, 2024-09-12T09:30:19.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59513 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140101+0000", 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140102+0000", 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9400000000000004e-07 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140106+0000", 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9269999999999998e-06 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140118+0000", 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140131+0000", 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3515e-05 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213318+0000", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073187167999999997 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213409+0000", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1179000000000002e-05 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213726+0000", 2024-09-12T09:30:19.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031658800000000003 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59514 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 20971520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5373136220000001, 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073929167000000004, 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59514 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140146+0000", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-07 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140150+0000", 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.963e-06 2024-09-12T09:30:19.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140162+0000", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1615e-05 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140172+0000", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213774+0000", 2024-09-12T09:30:19.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073601672000000007 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213848+0000", 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4418999999999993e-05 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214075+0000", 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022613399999999999 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59515 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21069824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5372427689999997, 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074140521000000001, 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59515 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140217+0000", 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140220+0000", 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1860000000000001e-06 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140231+0000", 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0570999999999999e-05 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140237+0000", 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214118+0000", 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073880571000000006 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214167+0000", 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9271999999999999e-05 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214357+0000", 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018998499999999999 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5372123110000002, 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:19.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:19.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5371898929999999, 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:19.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:19.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.5371667520000001, 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:19.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.537141793, 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:19.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:19.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:19.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:19.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:19.781 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:19.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:19.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:30:19.783 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:19.776+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:19.883 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:19.876+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:19.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:19.912 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone15 intact 2024-09-12T09:30:19.915 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:19.915 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:19.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:19.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:19.984 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:19.984+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:20.053 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:20.053 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:20.053 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:20.053 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:20.053 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:20.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:20.080 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:20.085 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:20.080+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:20.097 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:20.115 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:20.131 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:20.153 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-12T09:30:20.153 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:20.153 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:20.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:20.171 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:20.186 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:20.180+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:20.191 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:20.191 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:20.212 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:20.234 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:20.235 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:20.249 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:20.296 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:20.296+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:20.388 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:20.384+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:20.420 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:20.420 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:20.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:20.489 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:20.484+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:20.590 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:20.584+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:20.638 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:20.638 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:20.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:20.700 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:20.700+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:30:20.791 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:20.792+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:20.801 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:20.801 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:20.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:20.892 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:20.892+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:20.958 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:20.959 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:20.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:21.024 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:21.020+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:21.088 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:21.088 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:21.094 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:21.088+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:21.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:21.152 INFO:teuthology.orchestra.run.smithi007.stdout:1441 punch from 0x18bfe1a to 0x18cd904, (0xdaea bytes) 2024-09-12T09:30:21.163 INFO:teuthology.orchestra.run.smithi007.stdout:1442 trunc from 0xed3ff1d to 0xacfe46a 2024-09-12T09:30:21.194 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:21.188+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:21.232 INFO:teuthology.orchestra.run.smithi007.stdout:1445 write 0x7a3b89e thru 0x7a4b3d8 (0xfb3b bytes) 2024-09-12T09:30:21.264 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:21.264 INFO:teuthology.orchestra.run.smithi007.stdout:1449 read 0x2a08dfd thru 0x2a0d49b (0x469f bytes) 2024-09-12T09:30:21.264 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:21.273 INFO:teuthology.orchestra.run.smithi007.stdout:1451 write 0xd25eeb4 thru 0xd26dbe9 (0xed36 bytes) 2024-09-12T09:30:21.279 INFO:teuthology.orchestra.run.smithi007.stdout:1453 read 0x6bfe6c6 thru 0x6c0d7c1 (0xf0fc bytes) 2024-09-12T09:30:21.280 INFO:teuthology.orchestra.run.smithi007.stdout:1454 punch from 0x2a9d646 to 0x2aa6e0a, (0x97c4 bytes) 2024-09-12T09:30:21.280 INFO:teuthology.orchestra.run.smithi007.stdout:1456 read 0xb255cc7 thru 0xb259c15 (0x3f4f bytes) 2024-09-12T09:30:21.281 INFO:teuthology.orchestra.run.smithi007.stdout:1457 write 0x2eb049d thru 0x2eb1fa8 (0x1b0c bytes) 2024-09-12T09:30:21.295 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:21.292+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:21.299 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:21.299 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:21.305 INFO:teuthology.orchestra.run.smithi007.stdout:1458 read 0x2ede690 thru 0x2eebcd3 (0xd644 bytes) 2024-09-12T09:30:21.307 INFO:teuthology.orchestra.run.smithi007.stdout:1461 writesame 0x54821ef thru 0x548fa59 data_size 0x1(0xd86b bytes) 2024-09-12T09:30:21.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:21.349 INFO:teuthology.orchestra.run.smithi007.stdout:1463 writesame 0xdf1dcaa thru 0xdf2d438 data_size 0xa9(0xf78f bytes) 2024-09-12T09:30:21.351 INFO:teuthology.orchestra.run.smithi007.stdout:1464 read 0x823fb91 thru 0x8247b7b (0x7feb bytes) 2024-09-12T09:30:21.352 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:21.352 INFO:teuthology.orchestra.run.smithi007.stdout:1466 read 0x638de77 thru 0x63926a2 (0x482c bytes) 2024-09-12T09:30:21.352 INFO:teuthology.orchestra.run.smithi007.stdout:1467 read 0xab7e829 thru 0xab8064c (0x1e24 bytes) 2024-09-12T09:30:21.353 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:21.353 INFO:teuthology.orchestra.run.smithi007.stdout:1469 read 0x267e3c4 thru 0x2681f51 (0x3b8e bytes) 2024-09-12T09:30:21.354 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:21.354 INFO:teuthology.orchestra.run.smithi007.stdout:1471 read 0x59ad4ff thru 0x59b7784 (0xa286 bytes) 2024-09-12T09:30:21.355 INFO:teuthology.orchestra.run.smithi007.stdout:1473 read 0x69ce651 thru 0x69d795c (0x930c bytes) 2024-09-12T09:30:21.356 INFO:teuthology.orchestra.run.smithi007.stdout:1474 clone 17 order 25 su 524288 sc 8 2024-09-12T09:30:21.396 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:21.392+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:21.457 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:21.457 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:21.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:21.512 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:21.508+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:21.598 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:21.592+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:21.634 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:21.634 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:21.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:21.732 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:21.728+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:21.799 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:21.800+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:21.815 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:21.815 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:21.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:21.901 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:21.900+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:21.980 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone16 from 0xdf2d439 (overlap 0xacfe46a) to 0x67cbc3f 2024-09-12T09:30:21.986 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [6, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:21.986 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:21.986 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:30:21.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:30:22.001 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:22.000+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:22.019 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:22.019 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:22.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:22.132 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:22.128+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:22.203 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:22.196+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:22.213 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:22.213 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:22.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:22.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:22.304 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:22.304+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:22.329 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-12T09:30:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:22.342 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 63, 2024-09-12T09:30:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:30:22.342 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:30:21.772563+0000", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:29:16.082807+0000", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:30:05.649670+0000", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5801984, 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:22.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.344 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:22.345 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:22.346 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:22.347 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:22.348 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:22.349 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-12T09:30:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:30:22.351 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:22.352 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:22.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:22.355 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:22.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:22.358 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:22.359 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "63", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 19, 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 63, 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:22.360 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:22.361 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:22.362 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.25, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.25, 2024-09-12T09:30:22.363 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.690000057220459, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.690000057220459, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.364 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 56, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.367 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 51, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.371 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:30:22.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 51, 2024-09-12T09:30:22.377 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:30:22.379 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 54, 2024-09-12T09:30:22.383 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:30:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 54, 2024-09-12T09:30:22.389 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:30:22.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:30:22.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:30:22.394 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:22.395 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 56, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:30:22.397 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:30:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:30:22.401 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:30:22.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.406 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:22.407 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 46, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:22.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.410 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:30:22.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:22.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.415 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:30:22.416 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.418 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:22.419 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:30:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:30:22.421 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:22.422 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:22.424 INFO:tasks.thrashosds.thrasher:j is {'epoch': 63, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:30:21.772563+0000', 'last_up_change': '2024-09-12T09:29:16.082807+0000', 'last_in_change': '2024-09-12T09:30:05.649670+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 0.75, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:30:22.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:30:22.425 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:22.425 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:22.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:22.426 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:22.420+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:22.428 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:22.450 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:22.451 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:22.468 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:22.486 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:22.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:22.507 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:22.524 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:22.524+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:22.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:22.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:22.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:22.607 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:22.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:22.625 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:22.624+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:22.725 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:22.720+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:22.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:22.748 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:30:22.767 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-09-12T09:30:21.983250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":897443455,"num_objects":822,"num_object_clones":161,"num_object_copies":1644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":822,"num_whiteouts":382,"num_read":118683,"num_read_kb":847552,"num_write":5292,"num_write_kb":463022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":156314802,"num_keys_recovered":51,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4156,"ondisk_log_size":4156,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":274760,"kb_used_data":54344,"kb_used_omap":9,"kb_used_meta":220278,"kb_avail":565956280,"statfs":{"total":579820584960,"available":579539230720,"internally_reserved":0,"allocated":55648256,"data_stored":781202849,"data_compressed":4525035,"data_compressed_allocated":46256128,"data_compressed_original":775646071,"omap_allocated":9568,"internal_metadata":225565344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,37,38],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":130119181,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":33781,"num_read_kb":94341,"num_write":579,"num_write_kb":15962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.228415"},"pg_stats":[{"pgid":"3.1f","version":"60'340","reported_seq":3302,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.257704+0000","last_change":"2024-09-12T09:29:58.167241+0000","last_active":"2024-09-12T09:30:20.257704+0000","last_peered":"2024-09-12T09:30:20.257704+0000","last_clean":"2024-09-12T09:30:20.257704+0000","last_became_active":"2024-09-12T09:29:58.165903+0000","last_became_peered":"2024-09-12T09:29:58.165903+0000","last_unstale":"2024-09-12T09:30:20.257704+0000","last_undegraded":"2024-09-12T09:30:20.257704+0000","last_fullsized":"2024-09-12T09:30:20.257704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:05:59.120548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":64435112,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":4094,"num_read_kb":67898,"num_write":518,"num_write_kb":35556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":4329673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'101","reported_seq":1304,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.840391+0000","last_change":"2024-09-12T09:30:01.194793+0000","last_active":"2024-09-12T09:30:20.840391+0000","last_peered":"2024-09-12T09:30:20.840391+0000","last_clean":"2024-09-12T09:30:20.840391+0000","last_became_active":"2024-09-12T09:30:01.169430+0000","last_became_peered":"2024-09-12T09:30:01.169430+0000","last_unstale":"2024-09-12T09:30:20.840391+0000","last_undegraded":"2024-09-12T09:30:20.840391+0000","last_fullsized":"2024-09-12T09:30:20.840391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:36:27.119500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":31784598,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":1403,"num_read_kb":6405,"num_write":92,"num_write_kb":11214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":352930,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"58'107","reported_seq":2863,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837784+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:30:20.837784+0000","last_peered":"2024-09-12T09:30:20.837784+0000","last_clean":"2024-09-12T09:30:20.837784+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:30:20.837784+0000","last_undegraded":"2024-09-12T09:30:20.837784+0000","last_fullsized":"2024-09-12T09:30:20.837784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:56:05.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":37230621,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":2697,"num_read_kb":63096,"num_write":98,"num_write_kb":35581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"61'113","reported_seq":2608,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837784+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:30:20.837784+0000","last_peered":"2024-09-12T09:30:20.837784+0000","last_clean":"2024-09-12T09:30:20.837784+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:30:20.837784+0000","last_undegraded":"2024-09-12T09:30:20.837784+0000","last_fullsized":"2024-09-12T09:30:20.837784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:55:22.327361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":19732589,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2647,"num_read_kb":25777,"num_write":128,"num_write_kb":11336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"61'71","reported_seq":3452,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.726763+0000","last_change":"2024-09-12T09:29:55.231536+0000","last_active":"2024-09-12T09:30:20.726763+0000","last_peered":"2024-09-12T09:30:20.726763+0000","last_clean":"2024-09-12T09:30:20.726763+0000","last_became_active":"2024-09-12T09:29:55.137238+0000","last_became_peered":"2024-09-12T09:29:55.137238+0000","last_unstale":"2024-09-12T09:30:20.726763+0000","last_undegraded":"2024-09-12T09:30:20.726763+0000","last_fullsized":"2024-09-12T09:30:20.726763+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:20:04.128929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":12757441,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":3414,"num_read_kb":2284,"num_write":67,"num_write_kb":2577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1206936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"59'89","reported_seq":1011,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.709616+0000","last_change":"2024-09-12T09:29:36.673515+0000","last_active":"2024-09-12T09:30:16.709616+0000","last_peered":"2024-09-12T09:30:16.709616+0000","last_clean":"2024-09-12T09:30:16.709616+0000","last_became_active":"2024-09-12T09:29:34.665723+0000","last_became_peered":"2024-09-12T09:29:34.665723+0000","last_unstale":"2024-09-12T09:30:16.709616+0000","last_undegraded":"2024-09-12T09:30:16.709616+0000","last_fullsized":"2024-09-12T09:30:16.709616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:54:12.918288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":8311250,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":863,"num_read_kb":5524,"num_write":64,"num_write_kb":29039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"61'62","reported_seq":319,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.773961+0000","last_change":"2024-09-12T09:29:36.680185+0000","last_active":"2024-09-12T09:30:20.773961+0000","last_peered":"2024-09-12T09:30:20.773961+0000","last_clean":"2024-09-12T09:30:20.773961+0000","last_became_active":"2024-09-12T09:29:34.667496+0000","last_became_peered":"2024-09-12T09:29:34.667496+0000","last_unstale":"2024-09-12T09:30:20.773961+0000","last_undegraded":"2024-09-12T09:30:20.773961+0000","last_fullsized":"2024-09-12T09:30:20.773961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:39.914803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":13597632,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":398,"num_read_kb":1750,"num_write":49,"num_write_kb":1030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"58'70","reported_seq":529,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.840306+0000","last_change":"2024-09-12T09:29:55.177304+0000","last_active":"2024-09-12T09:30:20.840306+0000","last_peered":"2024-09-12T09:30:20.840306+0000","last_clean":"2024-09-12T09:30:20.840306+0000","last_became_active":"2024-09-12T09:29:55.134985+0000","last_became_peered":"2024-09-12T09:29:55.134985+0000","last_unstale":"2024-09-12T09:30:20.840306+0000","last_undegraded":"2024-09-12T09:30:20.840306+0000","last_fullsized":"2024-09-12T09:30:20.840306+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:18.966235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":5069729,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":332,"num_read_kb":3189,"num_write":60,"num_write_kb":4210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":524806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'71","reported_seq":1048,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.256730+0000","last_change":"2024-09-12T09:29:55.167682+0000","last_active":"2024-09-12T09:30:20.256730+0000","last_peered":"2024-09-12T09:30:20.256730+0000","last_clean":"2024-09-12T09:30:20.256730+0000","last_became_active":"2024-09-12T09:29:55.136789+0000","last_became_peered":"2024-09-12T09:29:55.136789+0000","last_unstale":"2024-09-12T09:30:20.256730+0000","last_undegraded":"2024-09-12T09:30:20.256730+0000","last_fullsized":"2024-09-12T09:30:20.256730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:49:42.344821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":25828235,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1320,"num_read_kb":27443,"num_write":148,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8413,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'156","reported_seq":12069,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837755+0000","last_change":"2024-09-12T09:30:06.784873+0000","last_active":"2024-09-12T09:30:20.837755+0000","last_peered":"2024-09-12T09:30:20.837755+0000","last_clean":"2024-09-12T09:30:20.837755+0000","last_became_active":"2024-09-12T09:30:06.668824+0000","last_became_peered":"2024-09-12T09:30:06.668824+0000","last_unstale":"2024-09-12T09:30:20.837755+0000","last_undegraded":"2024-09-12T09:30:20.837755+0000","last_fullsized":"2024-09-12T09:30:20.837755+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:58:31.936842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":39701644,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":12203,"num_read_kb":60748,"num_write":209,"num_write_kb":3846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2239600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"59'166","reported_seq":4552,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.840520+0000","last_change":"2024-09-12T09:30:09.705274+0000","last_active":"2024-09-12T09:30:20.840520+0000","last_peered":"2024-09-12T09:30:20.840520+0000","last_clean":"2024-09-12T09:30:20.840520+0000","last_became_active":"2024-09-12T09:30:09.704072+0000","last_became_peered":"2024-09-12T09:30:09.704072+0000","last_unstale":"2024-09-12T09:30:20.840520+0000","last_undegraded":"2024-09-12T09:30:20.840520+0000","last_fullsized":"2024-09-12T09:30:20.840520+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:58.800758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":63988982,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":4195,"num_read_kb":68240,"num_write":211,"num_write_kb":37067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":29632415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"59'404","reported_seq":4020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:17.006737+0000","last_change":"2024-09-12T09:30:09.704759+0000","last_active":"2024-09-12T09:30:17.006737+0000","last_peered":"2024-09-12T09:30:17.006737+0000","last_clean":"2024-09-12T09:30:17.006737+0000","last_became_active":"2024-09-12T09:30:09.703458+0000","last_became_peered":"2024-09-12T09:30:09.703458+0000","last_unstale":"2024-09-12T09:30:17.006737+0000","last_undegraded":"2024-09-12T09:30:17.006737+0000","last_fullsized":"2024-09-12T09:30:17.006737+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:05:37.077601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":16924694,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":5147,"num_read_kb":9622,"num_write":686,"num_write_kb":2531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":693960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.774967+0000","last_change":"2024-09-12T09:30:06.679265+0000","last_active":"2024-09-12T09:30:20.774967+0000","last_peered":"2024-09-12T09:30:20.774967+0000","last_clean":"2024-09-12T09:30:20.774967+0000","last_became_active":"2024-09-12T09:30:06.678386+0000","last_became_peered":"2024-09-12T09:30:06.678386+0000","last_unstale":"2024-09-12T09:30:20.774967+0000","last_undegraded":"2024-09-12T09:30:20.774967+0000","last_fullsized":"2024-09-12T09:30:20.774967+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:14:53.013801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"60'44","reported_seq":3588,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.719096+0000","last_change":"2024-09-12T09:29:55.208992+0000","last_active":"2024-09-12T09:30:20.719096+0000","last_peered":"2024-09-12T09:30:20.719096+0000","last_clean":"2024-09-12T09:30:20.719096+0000","last_became_active":"2024-09-12T09:29:55.138408+0000","last_became_peered":"2024-09-12T09:29:55.138408+0000","last_unstale":"2024-09-12T09:30:20.719096+0000","last_undegraded":"2024-09-12T09:30:20.719096+0000","last_fullsized":"2024-09-12T09:30:20.719096+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:12.127638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":12054579,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":3507,"num_read_kb":3170,"num_write":32,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":21350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.774127+0000","last_change":"2024-09-12T09:30:01.176975+0000","last_active":"2024-09-12T09:30:20.774127+0000","last_peered":"2024-09-12T09:30:20.774127+0000","last_clean":"2024-09-12T09:30:20.774127+0000","last_became_active":"2024-09-12T09:30:01.176687+0000","last_became_peered":"2024-09-12T09:30:01.176687+0000","last_unstale":"2024-09-12T09:30:20.774127+0000","last_undegraded":"2024-09-12T09:30:20.774127+0000","last_fullsized":"2024-09-12T09:30:20.774127+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:50:56.303434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"61'61","reported_seq":1286,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.256935+0000","last_change":"2024-09-12T09:30:12.711521+0000","last_active":"2024-09-12T09:30:20.256935+0000","last_peered":"2024-09-12T09:30:20.256935+0000","last_clean":"2024-09-12T09:30:20.256935+0000","last_became_active":"2024-09-12T09:30:12.711067+0000","last_became_peered":"2024-09-12T09:30:12.711067+0000","last_unstale":"2024-09-12T09:30:20.256935+0000","last_undegraded":"2024-09-12T09:30:20.256935+0000","last_fullsized":"2024-09-12T09:30:20.256935+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:29.565234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":19247120,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":1142,"num_read_kb":8037,"num_write":45,"num_write_kb":2891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:19.787215+0000","last_change":"2024-09-12T09:30:06.672902+0000","last_active":"2024-09-12T09:30:19.787215+0000","last_peered":"2024-09-12T09:30:19.787215+0000","last_clean":"2024-09-12T09:30:19.787215+0000","last_became_active":"2024-09-12T09:30:06.672028+0000","last_became_peered":"2024-09-12T09:30:06.672028+0000","last_unstale":"2024-09-12T09:30:19.787215+0000","last_undegraded":"2024-09-12T09:30:19.787215+0000","last_fullsized":"2024-09-12T09:30:19.787215+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:38.162810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"59'51","reported_seq":3269,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.773939+0000","last_change":"2024-09-12T09:29:36.680262+0000","last_active":"2024-09-12T09:30:20.773939+0000","last_peered":"2024-09-12T09:30:20.773939+0000","last_clean":"2024-09-12T09:30:20.773939+0000","last_became_active":"2024-09-12T09:29:34.667442+0000","last_became_peered":"2024-09-12T09:29:34.667442+0000","last_unstale":"2024-09-12T09:30:20.773939+0000","last_undegraded":"2024-09-12T09:30:20.773939+0000","last_fullsized":"2024-09-12T09:30:20.773939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:28:59.193473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":2991471,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":3132,"num_read_kb":3859,"num_write":44,"num_write_kb":1955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.774044+0000","last_change":"2024-09-12T09:29:55.138283+0000","last_active":"2024-09-12T09:30:20.774044+0000","last_peered":"2024-09-12T09:30:20.774044+0000","last_clean":"2024-09-12T09:30:20.774044+0000","last_became_active":"2024-09-12T09:29:55.137856+0000","last_became_peered":"2024-09-12T09:29:55.137856+0000","last_unstale":"2024-09-12T09:30:20.774044+0000","last_undegraded":"2024-09-12T09:30:20.774044+0000","last_fullsized":"2024-09-12T09:30:20.774044+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:50:58.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"59'92","reported_seq":12736,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.851855+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:30:20.851855+0000","last_peered":"2024-09-12T09:30:20.851855+0000","last_clean":"2024-09-12T09:30:20.851855+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:30:20.851855+0000","last_undegraded":"2024-09-12T09:30:20.851855+0000","last_fullsized":"2024-09-12T09:30:20.851855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:56:43.458010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":25542884,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":12817,"num_read_kb":50617,"num_write":74,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"59'176","reported_seq":3226,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.803846+0000","last_change":"2024-09-12T09:29:55.204077+0000","last_active":"2024-09-12T09:30:16.803846+0000","last_peered":"2024-09-12T09:30:16.803846+0000","last_clean":"2024-09-12T09:30:16.803846+0000","last_became_active":"2024-09-12T09:29:55.135753+0000","last_became_peered":"2024-09-12T09:29:55.135753+0000","last_unstale":"2024-09-12T09:30:16.803846+0000","last_undegraded":"2024-09-12T09:30:16.803846+0000","last_fullsized":"2024-09-12T09:30:16.803846+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:37.363868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":16118548,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":3819,"num_read_kb":13025,"num_write":169,"num_write_kb":7051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":57888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":103,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837678+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:30:20.837678+0000","last_peered":"2024-09-12T09:30:20.837678+0000","last_clean":"2024-09-12T09:30:20.837678+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:30:20.837678+0000","last_undegraded":"2024-09-12T09:30:20.837678+0000","last_fullsized":"2024-09-12T09:30:20.837678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"60'101","reported_seq":1620,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.774188+0000","last_change":"2024-09-12T09:29:36.679214+0000","last_active":"2024-09-12T09:30:20.774188+0000","last_peered":"2024-09-12T09:30:20.774188+0000","last_clean":"2024-09-12T09:30:20.774188+0000","last_became_active":"2024-09-12T09:29:34.667231+0000","last_became_peered":"2024-09-12T09:29:34.667231+0000","last_unstale":"2024-09-12T09:30:20.774188+0000","last_undegraded":"2024-09-12T09:30:20.774188+0000","last_fullsized":"2024-09-12T09:30:20.774188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:23.195329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":23481704,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":1471,"num_read_kb":11872,"num_write":85,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.773927+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:30:20.773927+0000","last_peered":"2024-09-12T09:30:20.773927+0000","last_clean":"2024-09-12T09:30:20.773927+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:30:20.773927+0000","last_undegraded":"2024-09-12T09:30:20.773927+0000","last_fullsized":"2024-09-12T09:30:20.773927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'149","reported_seq":9470,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.728802+0000","last_change":"2024-09-12T09:29:55.163561+0000","last_active":"2024-09-12T09:30:20.728802+0000","last_peered":"2024-09-12T09:30:20.728802+0000","last_clean":"2024-09-12T09:30:20.728802+0000","last_became_active":"2024-09-12T09:29:55.134480+0000","last_became_peered":"2024-09-12T09:29:55.134480+0000","last_unstale":"2024-09-12T09:30:20.728802+0000","last_undegraded":"2024-09-12T09:30:20.728802+0000","last_fullsized":"2024-09-12T09:30:20.728802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:36:22.781217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":36625208,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":9888,"num_read_kb":31623,"num_write":124,"num_write_kb":19323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2206098,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.774238+0000","last_change":"2024-09-12T09:30:06.682289+0000","last_active":"2024-09-12T09:30:20.774238+0000","last_peered":"2024-09-12T09:30:20.774238+0000","last_clean":"2024-09-12T09:30:20.774238+0000","last_became_active":"2024-09-12T09:30:06.681178+0000","last_became_peered":"2024-09-12T09:30:06.681178+0000","last_unstale":"2024-09-12T09:30:20.774238+0000","last_undegraded":"2024-09-12T09:30:20.774238+0000","last_fullsized":"2024-09-12T09:30:20.774238+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:48:19.258448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"62'86","reported_seq":3819,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.845829+0000","last_change":"2024-09-12T09:30:12.711941+0000","last_active":"2024-09-12T09:30:20.845829+0000","last_peered":"2024-09-12T09:30:20.845829+0000","last_clean":"2024-09-12T09:30:20.845829+0000","last_became_active":"2024-09-12T09:30:12.711307+0000","last_became_peered":"2024-09-12T09:30:12.711307+0000","last_unstale":"2024-09-12T09:30:20.845829+0000","last_undegraded":"2024-09-12T09:30:20.845829+0000","last_fullsized":"2024-09-12T09:30:20.845829+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:50.183393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":29564344,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":3922,"num_read_kb":31280,"num_write":105,"num_write_kb":10018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":102,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837669+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:30:20.837669+0000","last_peered":"2024-09-12T09:30:20.837669+0000","last_clean":"2024-09-12T09:30:20.837669+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:30:20.837669+0000","last_undegraded":"2024-09-12T09:30:20.837669+0000","last_fullsized":"2024-09-12T09:30:20.837669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":198,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:19.787065+0000","last_change":"2024-09-12T09:30:06.795347+0000","last_active":"2024-09-12T09:30:19.787065+0000","last_peered":"2024-09-12T09:30:19.787065+0000","last_clean":"2024-09-12T09:30:19.787065+0000","last_became_active":"2024-09-12T09:30:06.687237+0000","last_became_peered":"2024-09-12T09:30:06.687237+0000","last_unstale":"2024-09-12T09:30:19.787065+0000","last_undegraded":"2024-09-12T09:30:19.787065+0000","last_fullsized":"2024-09-12T09:30:19.787065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:37:26.434095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"61'206","reported_seq":2689,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.840356+0000","last_change":"2024-09-12T09:29:36.673435+0000","last_active":"2024-09-12T09:30:20.840356+0000","last_peered":"2024-09-12T09:30:20.840356+0000","last_clean":"2024-09-12T09:30:20.840356+0000","last_became_active":"2024-09-12T09:29:34.665426+0000","last_became_peered":"2024-09-12T09:29:34.665426+0000","last_unstale":"2024-09-12T09:30:20.840356+0000","last_undegraded":"2024-09-12T09:30:20.840356+0000","last_fullsized":"2024-09-12T09:30:20.840356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:18.644408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":35392808,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":3948,"num_read_kb":30548,"num_write":259,"num_write_kb":20368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"59'88","reported_seq":1065,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.414807+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:30:16.414807+0000","last_peered":"2024-09-12T09:30:16.414807+0000","last_clean":"2024-09-12T09:30:16.414807+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:30:16.414807+0000","last_undegraded":"2024-09-12T09:30:16.414807+0000","last_fullsized":"2024-09-12T09:30:16.414807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:50:04.949463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":8758125,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":904,"num_read_kb":2571,"num_write":74,"num_write_kb":7143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'88","reported_seq":2691,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.992737+0000","last_change":"2024-09-12T09:30:01.232115+0000","last_active":"2024-09-12T09:30:16.992737+0000","last_peered":"2024-09-12T09:30:16.992737+0000","last_clean":"2024-09-12T09:30:16.992737+0000","last_became_active":"2024-09-12T09:30:01.179562+0000","last_became_peered":"2024-09-12T09:30:01.179562+0000","last_unstale":"2024-09-12T09:30:16.992737+0000","last_undegraded":"2024-09-12T09:30:16.992737+0000","last_fullsized":"2024-09-12T09:30:16.992737+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:18.375840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":17487663,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":2629,"num_read_kb":6355,"num_write":133,"num_write_kb":4061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2670405,"num_keys_recovered":16,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"60'216","reported_seq":5194,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.840365+0000","last_change":"2024-09-12T09:29:55.239142+0000","last_active":"2024-09-12T09:30:20.840365+0000","last_peered":"2024-09-12T09:30:20.840365+0000","last_clean":"2024-09-12T09:30:20.840365+0000","last_became_active":"2024-09-12T09:29:55.135093+0000","last_became_peered":"2024-09-12T09:29:55.135093+0000","last_unstale":"2024-09-12T09:30:20.840365+0000","last_undegraded":"2024-09-12T09:30:20.840365+0000","last_fullsized":"2024-09-12T09:30:20.840365+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:53:25.444724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":60325389,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":5623,"num_read_kb":68262,"num_write":399,"num_write_kb":29356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1338871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"60'119","reported_seq":671,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.837844+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:30:20.837844+0000","last_peered":"2024-09-12T09:30:20.837844+0000","last_clean":"2024-09-12T09:30:20.837844+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:30:20.837844+0000","last_undegraded":"2024-09-12T09:30:20.837844+0000","last_fullsized":"2024-09-12T09:30:20.837844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:53:24.035833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":13480245,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":1314,"num_read_kb":4530,"num_write":113,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'120","reported_seq":5890,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.849725+0000","last_change":"2024-09-12T09:30:06.830214+0000","last_active":"2024-09-12T09:30:20.849725+0000","last_peered":"2024-09-12T09:30:20.849725+0000","last_clean":"2024-09-12T09:30:20.849725+0000","last_became_active":"2024-09-12T09:30:06.685061+0000","last_became_peered":"2024-09-12T09:30:06.685061+0000","last_unstale":"2024-09-12T09:30:20.849725+0000","last_undegraded":"2024-09-12T09:30:20.849725+0000","last_fullsized":"2024-09-12T09:30:20.849725+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:54.628545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":19169678,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":6235,"num_read_kb":21453,"num_write":189,"num_write_kb":1930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":557161,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"57'118","reported_seq":2015,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:15.751023+0000","last_change":"2024-09-12T09:30:09.708314+0000","last_active":"2024-09-12T09:30:15.751023+0000","last_peered":"2024-09-12T09:30:15.751023+0000","last_clean":"2024-09-12T09:30:15.751023+0000","last_became_active":"2024-09-12T09:30:09.706935+0000","last_became_peered":"2024-09-12T09:30:09.706935+0000","last_unstale":"2024-09-12T09:30:15.751023+0000","last_undegraded":"2024-09-12T09:30:15.751023+0000","last_fullsized":"2024-09-12T09:30:15.751023+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:39.591930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":42191909,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":2200,"num_read_kb":58158,"num_write":164,"num_write_kb":38323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":25316217,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"59'145","reported_seq":4177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.863324+0000","last_change":"2024-09-12T09:30:09.705263+0000","last_active":"2024-09-12T09:30:16.863324+0000","last_peered":"2024-09-12T09:30:16.863324+0000","last_clean":"2024-09-12T09:30:16.863324+0000","last_became_active":"2024-09-12T09:30:09.704611+0000","last_became_peered":"2024-09-12T09:30:09.704611+0000","last_unstale":"2024-09-12T09:30:16.863324+0000","last_undegraded":"2024-09-12T09:30:16.863324+0000","last_fullsized":"2024-09-12T09:30:16.863324+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:25.401243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":46672051,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":4239,"num_read_kb":58547,"num_write":200,"num_write_kb":36336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":34086545,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"59'152","reported_seq":1049,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.256946+0000","last_change":"2024-09-12T09:30:03.657130+0000","last_active":"2024-09-12T09:30:20.256946+0000","last_peered":"2024-09-12T09:30:20.256946+0000","last_clean":"2024-09-12T09:30:20.256946+0000","last_became_active":"2024-09-12T09:30:03.655757+0000","last_became_peered":"2024-09-12T09:30:03.655757+0000","last_unstale":"2024-09-12T09:30:20.256946+0000","last_undegraded":"2024-09-12T09:30:20.256946+0000","last_fullsized":"2024-09-12T09:30:20.256946+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:06.747428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":26015975,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":1037,"num_read_kb":6021,"num_write":179,"num_write_kb":11663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":4010639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"59'63","reported_seq":1329,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.589937+0000","last_change":"2024-09-12T09:30:06.738560+0000","last_active":"2024-09-12T09:30:16.589937+0000","last_peered":"2024-09-12T09:30:16.589937+0000","last_clean":"2024-09-12T09:30:16.589937+0000","last_became_active":"2024-09-12T09:30:06.679491+0000","last_became_peered":"2024-09-12T09:30:06.679491+0000","last_unstale":"2024-09-12T09:30:16.589937+0000","last_undegraded":"2024-09-12T09:30:16.589937+0000","last_fullsized":"2024-09-12T09:30:16.589937+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:47.679436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":27111970,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":1205,"num_read_kb":7028,"num_write":84,"num_write_kb":669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":299330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"59'193","reported_seq":10018,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-12T09:30:20.730715+0000","last_change":"2024-09-12T09:30:09.705603+0000","last_active":"2024-09-12T09:30:20.730715+0000","last_peered":"2024-09-12T09:30:20.730715+0000","last_clean":"2024-09-12T09:30:20.730715+0000","last_became_active":"2024-09-12T09:30:09.703273+0000","last_became_peered":"2024-09-12T09:30:09.703273+0000","last_unstale":"2024-09-12T09:30:20.730715+0000","last_undegraded":"2024-09-12T09:30:20.730715+0000","last_fullsized":"2024-09-12T09:30:20.730715+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:35.380992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":62108607,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":10471,"num_read_kb":85171,"num_write":229,"num_write_kb":49183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":43473478,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'63","reported_seq":573,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-12T09:30:16.710684+0000","last_change":"2024-09-12T09:30:06.798226+0000","last_active":"2024-09-12T09:30:16.710684+0000","last_peered":"2024-09-12T09:30:16.710684+0000","last_clean":"2024-09-12T09:30:16.710684+0000","last_became_active":"2024-09-12T09:30:06.682301+0000","last_became_peered":"2024-09-12T09:30:06.682301+0000","last_unstale":"2024-09-12T09:30:16.710684+0000","last_undegraded":"2024-09-12T09:30:16.710684+0000","last_fullsized":"2024-09-12T09:30:16.710684+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:22.830334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":33150263,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":401,"num_read_kb":3394,"num_write":158,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2107351,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":896853068,"num_objects":818,"num_object_clones":161,"num_object_copies":1636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":818,"num_whiteouts":382,"num_read":118617,"num_read_kb":847497,"num_write":5189,"num_write_kb":461392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":155134066,"num_keys_recovered":51,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":60837888,"data_stored":912047927,"data_compressed":5157707,"data_compressed_allocated":55136256,"data_compressed_original":908018856,"omap_allocated":0,"internal_metadata":0},"log_size":4081,"ondisk_log_size":4081,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":2361472,"data_compressed":25801,"data_compressed_allocated":147456,"data_compressed_original":2342912,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509455,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48436,"kb_used_data":13092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323404,"statfs":{"total":96636764160,"available":96587165696,"internally_reserved":0,"allocated":13406208,"data_stored":200468157,"data_compressed":1132801,"data_compressed_allocated":11526144,"data_compressed_original":199265671,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,37,38],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542161,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43908,"kb_used_data":8552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327932,"statfs":{"total":96636764160,"available":96591802368,"internally_reserved":0,"allocated":8757248,"data_stored":123953500,"data_compressed":448672,"data_compressed_allocated":7249920,"data_compressed_original":123109376,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50612,"kb_used_data":11104,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321228,"statfs":{"total":96636764160,"available":96584937472,"internally_reserved":0,"allocated":11370496,"data_stored":165391099,"data_compressed":1050203,"data_compressed_allocated":9785344,"data_compressed_original":164514488,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37868,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333972,"statfs":{"total":96636764160,"available":96597987328,"internally_reserved":0,"allocated":2580480,"data_stored":10757273,"data_compressed":354153,"data_compressed_allocated":1540096,"data_compressed_original":10252288,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40864,"kb_used_data":5512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330976,"statfs":{"total":96636764160,"available":96594919424,"internally_reserved":0,"allocated":5644288,"data_stored":66022381,"data_compressed":486591,"data_compressed_allocated":4218880,"data_compressed_original":65154601,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53072,"kb_used_data":13564,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94318768,"statfs":{"total":96636764160,"available":96582418432,"internally_reserved":0,"allocated":13889536,"data_stored":214610439,"data_compressed":1052615,"data_compressed_allocated":11935744,"data_compressed_original":213349647,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5839,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13148160,"data_stored":213708385,"data_compressed":1045961,"data_compressed_allocated":11898880,"data_compressed_original":212759823,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4898816,"data_stored":65120308,"data_compressed":479937,"data_compressed_allocated":4182016,"data_compressed_original":64564777,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":10445587,"data_compressed":354153,"data_compressed_allocated":1540096,"data_compressed_original":10252288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10670080,"data_stored":165079413,"data_compressed":1050203,"data_compressed_allocated":9785344,"data_compressed_original":164514488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12730368,"data_stored":200168757,"data_compressed":1132801,"data_compressed_allocated":11526144,"data_compressed_original":199265671,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8052736,"data_stored":123641795,"data_compressed":448672,"data_compressed_allocated":7249920,"data_compressed_original":123109376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":4709750,"data_compressed":109990,"data_compressed_allocated":405504,"data_compressed_original":4681728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8990720,"data_stored":129173932,"data_compressed":535990,"data_compressed_allocated":8548352,"data_compressed_original":128870705,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:30:22.769 INFO:tasks.thrashosds.thrasher:Setting 3.e to [5, 0] 2024-09-12T09:30:22.769 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.e', '5', '0'] 2024-09-12T09:30:22.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.e 5 0 2024-09-12T09:30:22.775 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:22.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:30:22.794 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:22.795 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:22.798 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:22.799 INFO:teuthology.orchestra.run.smithi007.stdout:1476 write 0x3752aa5 thru 0x375e26a (0xb7c6 bytes) 2024-09-12T09:30:22.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:22.827 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:22.824+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:22.909 INFO:teuthology.orchestra.run.smithi007.stdout:1478 punch from 0x992a803 to 0x993683d, (0xc03a bytes) 2024-09-12T09:30:22.909 INFO:teuthology.orchestra.run.smithi007.stdout:1479 trunc from 0xdf2d439 to 0xe739056 2024-09-12T09:30:22.928 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:22.928+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:22.928 INFO:teuthology.orchestra.run.smithi007.stdout:1480 trunc from 0xe739056 to 0xaad45d5 2024-09-12T09:30:23.030 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:23.024+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:23.093 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53986 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 14909440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.704617403, 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055388721000000002, 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53986 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387069+0000", 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.905e-06 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387076+0000", 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387087+0000", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1499e-05 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442148+0000", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055060410999999997 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442229+0000", 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1441999999999997e-05 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442456+0000", 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000226534 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53987 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15007744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.704589, 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055709171000000002, 2024-09-12T09:30:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53987 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387096+0000", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387098+0000", 2024-09-12T09:30:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0770000000000002e-06 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387105+0000", 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7619999999999998e-06 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387113+0000", 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4959999999999996e-06 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442489+0000", 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055376140999999997 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442610+0000", 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120234 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442805+0000", 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019502899999999999 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53988 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15106048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.704563608000001, 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057056229, 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53988 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387123+0000", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387130+0000", 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5810000000000002e-06 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387135+0000", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1490000000000002e-06 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442836+0000", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055700889000000003 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442918+0000", 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000004e-05 2024-09-12T09:30:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.444177+0000", 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012590570000000001 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54006 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17367040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.70435719, 2024-09-12T09:30:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056089721000000002, 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54006 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389586+0000", 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002258552 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389632+0000", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5796e-05 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389644+0000", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2699e-05 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389667+0000", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3076999999999999e-05 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443194+0000", 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053526642999999999 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443320+0000", 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012563599999999999 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443417+0000", 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7318000000000006e-05 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54034 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17694720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.700594425, 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055444460000000001, 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54034 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391755+0000", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066475499999999995 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391760+0000", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.181e-06 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391904+0000", 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014413300000000001 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391916+0000", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1507e-05 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446235+0000", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054319522000000002 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446313+0000", 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7745999999999997e-05 2024-09-12T09:30:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446534+0000", 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221616 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54051 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20250624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.698923438, 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055664077999999999, 2024-09-12T09:30:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54051 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392762+0000", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392764+0000", 2024-09-12T09:30:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4710000000000001e-06 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392772+0000", 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6969999999999993e-06 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392777+0000", 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.587e-06 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447587+0000", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.05480952 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447661+0000", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4635000000000006e-05 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448425+0000", 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076352799999999997 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54052 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20348928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.698896717, 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056119461000000002, 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54052 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393045+0000", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025747600000000003 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393054+0000", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0159999999999997e-06 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393064+0000", 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4050000000000006e-06 2024-09-12T09:30:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393071+0000", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6290000000000001e-06 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448481+0000", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055409753999999999 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448576+0000", 2024-09-12T09:30:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4969999999999994e-05 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448907+0000", 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033121099999999998 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61223 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13336576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8521029920000007, 2024-09-12T09:30:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055340981999999997, 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61223 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239581+0000", 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239582+0000", 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3500000000000002e-07 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239584+0000", 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3860000000000001e-06 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239635+0000", 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0985000000000001e-05 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.239643+0000", 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8869999999999992e-06 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294604+0000", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054961083000000001 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294717+0000", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011312 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.294922+0000", 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020508599999999999 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61227 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13533184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8514309190000002, 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055556323999999997, 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61227 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240254+0000", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240257+0000", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.762e-06 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240265+0000", 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8469999999999996e-06 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240271+0000", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0689999999999998e-06 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295548+0000", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055277706000000003 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295630+0000", 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1450000000000001e-05 2024-09-12T09:30:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295810+0000", 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017996399999999999 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61229 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13631488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8514043420000004, 2024-09-12T09:30:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055868963000000001, 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61229 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240281+0000", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240283+0000", 2024-09-12T09:30:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240290+0000", 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0570000000000004e-06 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240295+0000", 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.164e-06 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295854+0000", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055558626999999999 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295952+0000", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8405000000000004e-05 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296149+0000", 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019671099999999999 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61231 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13729792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8513802409999993, 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056138067999999999, 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61231 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240305+0000", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240307+0000", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.294e-06 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240315+0000", 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1019999999999997e-06 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240321+0000", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9370000000000001e-06 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296180+0000", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055859345999999997 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296248+0000", 2024-09-12T09:30:23.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7596999999999997e-05 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296442+0000", 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019433699999999999 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61233 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13828096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:23.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8513554760000002, 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056534344, 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61233 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:23.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240332+0000", 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240340+0000", 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240346+0000", 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296492+0000", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056146433000000003 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296606+0000", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011335799999999999 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296863+0000", 2024-09-12T09:30:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000257612 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61235 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13926400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8513289610000001, 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056785587999999998, 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61235 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:23.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240356+0000", 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240358+0000", 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5390000000000002e-06 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240367+0000", 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6750000000000008e-06 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240372+0000", 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2789999999999998e-06 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296882+0000", 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056509707999999999 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296969+0000", 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7366000000000003e-05 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297141+0000", 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017161100000000001 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61237 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14024704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8513045090000002, 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057667507, 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61237 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9400000000000001e-07 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240382+0000", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1900000000000002e-06 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240389+0000", 2024-09-12T09:30:23.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4060000000000002e-06 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240394+0000", 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9300000000000002e-06 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297172+0000", 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056777785999999997 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297313+0000", 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014155099999999999 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298047+0000", 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073424999999999999 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61239 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8512823669999996, 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057981773, 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61239 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9000000000000002e-07 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240405+0000", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240412+0000", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.469e-06 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240427+0000", 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5102e-05 2024-09-12T09:30:23.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298080+0000", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057653049999999997 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298166+0000", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5296000000000005e-05 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298384+0000", 2024-09-12T09:30:23.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000218027 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61241 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14221312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8512463659999998, 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058244112000000001, 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61241 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240441+0000", 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7020000000000001e-06 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240450+0000", 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240455+0000", 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298403+0000", 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057947945000000001 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298473+0000", 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9863999999999994e-05 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298682+0000", 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020926199999999999 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.167 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61243 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14319616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8512213640000006, 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058591126, 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61243 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-12T09:30:23.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240466+0000", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7010000000000001e-06 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240473+0000", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240479+0000", 2024-09-12T09:30:23.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.327e-06 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298722+0000", 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058243504000000001 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298783+0000", 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1137999999999996e-05 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299054+0000", 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027089799999999999 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61245 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14417920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.851197591, 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058955593000000001, 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61245 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240494+0000", 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5420000000000002e-06 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240511+0000", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240518+0000", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5989999999999998e-06 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299110+0000", 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058592712999999998 2024-09-12T09:30:23.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299190+0000", 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9622000000000004e-05 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299442+0000", 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025237399999999998 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61247 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14516224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8511569170000008, 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059307199999999997, 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61247 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240528+0000", 2024-09-12T09:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240531+0000", 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.615e-06 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240539+0000", 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240545+0000", 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0310000000000004e-06 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299503+0000", 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058957756 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299625+0000", 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012207099999999999 2024-09-12T09:30:23.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299835+0000", 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209605 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61249 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14614528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8511306310000002, 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059607902999999997, 2024-09-12T09:30:23.181 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61249 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240557+0000", 2024-09-12T09:30:23.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5780000000000001e-06 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240565+0000", 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5040000000000002e-06 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240570+0000", 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0690000000000001e-06 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299884+0000", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059313728000000003 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299967+0000", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3404000000000003e-05 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.300162+0000", 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019417999999999999 2024-09-12T09:30:23.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:23.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:23.186 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:23.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:23.188 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:23.180+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:23.288 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:23.284+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:23.323 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.323 INFO:teuthology.orchestra.run.smithi007.stdout:1482 write 0xa8c1347 thru 0xa8c808d (0x6d47 bytes) 2024-09-12T09:30:23.325 INFO:teuthology.orchestra.run.smithi007.stdout:1484 writesame 0x88c83df thru 0x88cf9db data_size 0x7(0x75fd bytes) 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116949 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [stat out=16b,write-same in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.326887+0000", 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0016600339999999999, 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016972249999999999, 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:30:23.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116949 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326887+0000", 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326887+0000", 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326888+0000", 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1310000000000001e-06 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326905+0000", 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6932e-05 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326922+0000", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6538999999999999e-05 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326931+0000", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.326992+0000", 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1186000000000007e-05 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.327068+0000", 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5440999999999998e-05 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.327619+0000", 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055133899999999997 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.332 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:23.333 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:30:23.333 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:23.333 INFO:teuthology.orchestra.run.smithi007.stdout:1485 writesame 0xa680cb2 thru 0xa682fa4 data_size 0x1(0x22f3 bytes) 2024-09-12T09:30:23.334 INFO:teuthology.orchestra.run.smithi007.stdout:1486 punch from 0x5aa2447 to 0x5aa8f0d, (0x6ac6 bytes) 2024-09-12T09:30:23.335 INFO:teuthology.orchestra.run.smithi007.stdout:1487 punch from 0x2da3164 to 0x2da3fcb, (0xe67 bytes) 2024-09-12T09:30:23.335 INFO:teuthology.orchestra.run.smithi007.stdout:1488 write 0x2d27412 thru 0x2d33595 (0xc184 bytes) 2024-09-12T09:30:23.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:23.348 INFO:teuthology.orchestra.run.smithi007.stdout:1489 write 0xcbba784 thru 0xcbc77e7 (0xd064 bytes) 2024-09-12T09:30:23.362 INFO:teuthology.orchestra.run.smithi007.stdout:1490 trunc from 0xcbc77e8 to 0x335b9b2 2024-09-12T09:30:23.387 INFO:teuthology.orchestra.run.smithi007.stdout:1492 trunc from 0x335b9b2 to 0xbd10e86 2024-09-12T09:30:23.389 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:23.388+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:23.395 INFO:teuthology.orchestra.run.smithi007.stdout:1493 write 0x5f70e46 thru 0x5f7f87d (0xea38 bytes) 2024-09-12T09:30:23.398 INFO:teuthology.orchestra.run.smithi007.stdout:1494 read 0xb542cf6 thru 0xb545344 (0x264f bytes) 2024-09-12T09:30:23.398 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.398 INFO:teuthology.orchestra.run.smithi007.stdout:1497 punch from 0x2fe1ab2 to 0x2fedaca, (0xc018 bytes) 2024-09-12T09:30:23.398 INFO:teuthology.orchestra.run.smithi007.stdout:1498 write 0x5e5b421 thru 0x5e69c32 (0xe812 bytes) 2024-09-12T09:30:23.400 INFO:teuthology.orchestra.run.smithi007.stdout:1499 read 0x21579e0 thru 0x21639d6 (0xbff7 bytes) 2024-09-12T09:30:23.401 INFO:teuthology.orchestra.run.smithi007.stdout:1500 write 0x94b8548 thru 0x94c4251 (0xbd0a bytes) 2024-09-12T09:30:23.403 INFO:teuthology.orchestra.run.smithi007.stdout:1503 write 0x70c1678 thru 0x70c9077 (0x7a00 bytes) 2024-09-12T09:30:23.405 INFO:teuthology.orchestra.run.smithi007.stdout:1504 read 0x158a36b thru 0x15980bc (0xdd52 bytes) 2024-09-12T09:30:23.406 INFO:teuthology.orchestra.run.smithi007.stdout:1505 write 0xa28204a thru 0xa28c115 (0xa0cc bytes) 2024-09-12T09:30:23.408 INFO:teuthology.orchestra.run.smithi007.stdout:1506 write 0x93aee3e thru 0x93b8c8b (0x9e4e bytes) 2024-09-12T09:30:23.411 INFO:teuthology.orchestra.run.smithi007.stdout:1507 punch from 0x251c36a to 0x251f87b, (0x3511 bytes) 2024-09-12T09:30:23.411 INFO:teuthology.orchestra.run.smithi007.stdout:1508 write 0x43dd349 thru 0x43e745c (0xa114 bytes) 2024-09-12T09:30:23.413 INFO:teuthology.orchestra.run.smithi007.stdout:1512 read 0x1ae154a thru 0x1ae7290 (0x5d47 bytes) 2024-09-12T09:30:23.414 INFO:teuthology.orchestra.run.smithi007.stdout:1513 punch from 0xb29ca8f to 0xb29fa79, (0x2fea bytes) 2024-09-12T09:30:23.414 INFO:teuthology.orchestra.run.smithi007.stdout:1514 read 0x25a9923 thru 0x25b847d (0xeb5b bytes) 2024-09-12T09:30:23.414 INFO:teuthology.orchestra.run.smithi007.stdout:1515 write 0xa0ef6d6 thru 0xa0f8cc7 (0x95f2 bytes) 2024-09-12T09:30:23.417 INFO:teuthology.orchestra.run.smithi007.stdout:1516 trunc from 0xbd10e86 to 0xbe21901 2024-09-12T09:30:23.425 INFO:teuthology.orchestra.run.smithi007.stdout:1517 read 0x74325f5 thru 0x7440e68 (0xe874 bytes) 2024-09-12T09:30:23.426 INFO:teuthology.orchestra.run.smithi007.stdout:1519 write 0x1b0b thru 0xe3d2 (0xc8c8 bytes) 2024-09-12T09:30:23.435 INFO:teuthology.orchestra.run.smithi007.stdout:1521 trunc from 0xbe21901 to 0x7292002 2024-09-12T09:30:23.447 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:1523 read 0x3d59580 thru 0x3d620b7 (0x8b38 bytes) 2024-09-12T09:30:23.448 INFO:teuthology.orchestra.run.smithi007.stdout:1524 writesame 0x1418ee9 thru 0x141e824 data_size 0x2(0x593c bytes) 2024-09-12T09:30:23.458 INFO:teuthology.orchestra.run.smithi007.stdout:1525 read 0x4f2a97 thru 0x4f650f (0x3a79 bytes) 2024-09-12T09:30:23.459 INFO:teuthology.orchestra.run.smithi007.stdout:1526 punch from 0x5825af3 to 0x5833be0, (0xe0ed bytes) 2024-09-12T09:30:23.459 INFO:teuthology.orchestra.run.smithi007.stdout:1527 punch from 0x136303 to 0x137e71, (0x1b6e bytes) 2024-09-12T09:30:23.473 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:23.473 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:23.473 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:23.473 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:23.474 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:23.476 INFO:teuthology.orchestra.run.smithi007.stdout:1529 writesame 0xb576fea thru 0xb57cbc2 data_size 0x1(0x5bd9 bytes) 2024-09-12T09:30:23.477 INFO:teuthology.orchestra.run.smithi007.stdout:1530 read 0x9609a5c thru 0x960facb (0x6070 bytes) 2024-09-12T09:30:23.478 INFO:teuthology.orchestra.run.smithi007.stdout:1531 write 0x90e74bd thru 0x90ec367 (0x4eab bytes) 2024-09-12T09:30:23.481 INFO:teuthology.orchestra.run.smithi007.stdout:1532 write 0x7c68ff5 thru 0x7c710f9 (0x8105 bytes) 2024-09-12T09:30:23.488 INFO:teuthology.orchestra.run.smithi007.stdout:1534 trunc from 0xb57cbc3 to 0x8f85d37 2024-09-12T09:30:23.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:23.503 INFO:teuthology.orchestra.run.smithi007.stdout:1535 writesame 0xa6a431 thru 0xa75a20 data_size 0x52(0xb5f0 bytes) 2024-09-12T09:30:23.516 INFO:teuthology.orchestra.run.smithi007.stdout:1536 trunc from 0x8f85d37 to 0xe312e3c 2024-09-12T09:30:23.528 INFO:teuthology.orchestra.run.smithi007.stdout:1537 read 0x3e175d0 thru 0x3e1864c (0x107d bytes) 2024-09-12T09:30:23.528 INFO:teuthology.orchestra.run.smithi007.stdout:1538 writesame 0xdbf2c9f thru 0xdbf7eba data_size 0xa(0x521c bytes) 2024-09-12T09:30:23.532 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:23.532+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:1540 trunc from 0xe312e3c to 0x26c7daf 2024-09-12T09:30:23.569 INFO:teuthology.orchestra.run.smithi007.stdout:1541 punch from 0x247cf5b to 0x2483ce9, (0x6d8e bytes) 2024-09-12T09:30:23.569 INFO:teuthology.orchestra.run.smithi007.stdout:1542 writesame 0x1f867af thru 0x1f95751 data_size 0x27(0xefa3 bytes) 2024-09-12T09:30:23.572 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.573 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.573 INFO:teuthology.orchestra.run.smithi007.stdout:1545 read 0xf638ed thru 0xf73006 (0xf71a bytes) 2024-09-12T09:30:23.574 INFO:teuthology.orchestra.run.smithi007.stdout:1546 punch from 0x206e2ea to 0x2075f5b, (0x7c71 bytes) 2024-09-12T09:30:23.585 INFO:teuthology.orchestra.run.smithi007.stdout:1547 write 0x2a0ec76 thru 0x2a10a72 (0x1dfd bytes) 2024-09-12T09:30:23.591 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:23.584+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:23.620 INFO:teuthology.orchestra.run.smithi007.stdout:1550 writesame 0xab757c9 thru 0xab7d2a3 data_size 0x1(0x7adb bytes) 2024-09-12T09:30:23.623 INFO:teuthology.orchestra.run.smithi007.stdout:1551 punch from 0xf000fd to 0xf0a807, (0xa70a bytes) 2024-09-12T09:30:23.623 INFO:teuthology.orchestra.run.smithi007.stdout:1552 writesame 0x2242e3c thru 0x224985c data_size 0x1(0x6a21 bytes) 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.36362763, 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.655 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.904131401000001, 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:23.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.53071445, 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:23.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:23.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.530667203, 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:23.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:23.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.530374127, 2024-09-12T09:30:23.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:23.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:23.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.529812195, 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:23.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.529760832000001, 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.529187952000001, 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:23.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.529035652999999, 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:23.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.528678563, 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:23.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.528053592999999, 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:23.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:23.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:23.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:23.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.528002257000001, 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:23.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.527207467, 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:23.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.527132577, 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:23.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:23.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.527080257, 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:23.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:23.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.527031659, 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:23.729 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:23.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:23.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.526777193999999, 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:23.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58810 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14483456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.738 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.523374928999999, 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069618174000000005, 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58810 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129306+0000", 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129311+0000", 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129327+0000", 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6446000000000001e-05 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129338+0000", 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0865e-05 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198581+0000", 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069242608999999997 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198675+0000", 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4285999999999998e-05 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198923+0000", 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024778499999999999 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.523322904, 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:23.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:23.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.46575435700000001, 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:23.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:23.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:30:23.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:23.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.754 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:23.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1553 read 0x130ce48 thru 0x131387f (0x6a38 bytes) 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1555 write 0x423b393 thru 0x4242846 (0x74b4 bytes) 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1557 trunc from 0xab7d2a4 to 0x805cd7 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1559 read 0x1262fc thru 0x12c1f9 (0x5efe bytes) 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1561 trunc from 0x805cd7 to 0x373bc03 2024-09-12T09:30:23.757 INFO:teuthology.orchestra.run.smithi007.stdout:1563 read 0x371ffe4 thru 0x372730d (0x732a bytes) 2024-09-12T09:30:23.757 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:23.748+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:23.758 INFO:teuthology.orchestra.run.smithi007.stdout:1565 write 0x7376ce4 thru 0x737b1a5 (0x44c2 bytes) 2024-09-12T09:30:23.761 INFO:teuthology.orchestra.run.smithi007.stdout:1566 read 0x219fccf thru 0x21a1a6f (0x1da1 bytes) 2024-09-12T09:30:23.762 INFO:teuthology.orchestra.run.smithi007.stdout:1567 writesame 0x6e8d156 thru 0x6e8d42c data_size 0x1(0x2d7 bytes) 2024-09-12T09:30:23.765 INFO:teuthology.orchestra.run.smithi007.stdout:1568 read 0x3204526 thru 0x320c590 (0x806b bytes) 2024-09-12T09:30:23.766 INFO:teuthology.orchestra.run.smithi007.stdout:1569 write 0xe9f526 thru 0xea5a3b (0x6516 bytes) 2024-09-12T09:30:23.783 INFO:teuthology.orchestra.run.smithi007.stdout:1570 writesame 0x7cc0984 thru 0x7cc6fbb data_size 0x4(0x6638 bytes) 2024-09-12T09:30:23.786 INFO:teuthology.orchestra.run.smithi007.stdout:1573 write 0x155d57f thru 0x156cd2b (0xf7ad bytes) 2024-09-12T09:30:23.794 INFO:teuthology.orchestra.run.smithi007.stdout:1574 trunc from 0x7cc6fbc to 0x97de093 2024-09-12T09:30:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:30:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.e pg_upmap mapping to [5,0] 2024-09-12T09:30:23.856 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:23.856+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.091153814999999999, 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.091193076999999997, 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:30:23.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.091054524999999997, 2024-09-12T09:30:23.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.09119853, 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:23.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:30:23.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:23.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:30:23.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 2 2024-09-12T09:30:23.900 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:23.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:23.957 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:23.952+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:24.058 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:24.056+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:24.070 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:24.070 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:24.070 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:24.070 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:24.070 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:24.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:24.172 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:24.172+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.840559784999996, 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:24.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.840464523000001, 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:24.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.836779899, 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.263 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:24.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.834974809000002, 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.268 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:24.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:24.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.834428133999999, 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:24.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.833659691000001, 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:24.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.833249322999997, 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:24.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.833110146999999, 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:24.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:24.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:24.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.825331335999998, 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:24.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.298 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.822967321999997, 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:24.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:24.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.812215428999998, 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:24.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:24.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:24.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.812019038000003, 2024-09-12T09:30:24.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:24.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:24.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.810524618000002, 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59514 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 20971520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110383068000001, 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073929167000000004, 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59514 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140145+0000", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140146+0000", 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-07 2024-09-12T09:30:24.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140150+0000", 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.963e-06 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140162+0000", 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1615e-05 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140172+0000", 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0569e-05 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213774+0000", 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073601672000000007 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.213848+0000", 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4418999999999993e-05 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214075+0000", 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022613399999999999 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59515 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21069824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110312215, 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074140521000000001, 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59515 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140216+0000", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140217+0000", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140220+0000", 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1860000000000001e-06 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140231+0000", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0570999999999999e-05 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140237+0000", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214118+0000", 2024-09-12T09:30:24.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073880571000000006 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214167+0000", 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9271999999999999e-05 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214357+0000", 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018998499999999999 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110281756999999, 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:24.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:24.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110259339000001, 2024-09-12T09:30:24.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:24.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110236198000001, 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:24.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:24.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.110211239, 2024-09-12T09:30:24.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:24.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:24.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.066859725, 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:30:24.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.353 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:24.354 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:24.355 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:24.352+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:24.454 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:24.452+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:24.521 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:24.521 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:24.521 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:24.522 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:24.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:24.580 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:24.580+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:24.657 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:24.656+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:24.697 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:24.698 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:24.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:24.758 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:24.756+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:24.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:24.858 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:24.852+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:24.885 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:24.885 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:24.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:24.885 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246568680000003, 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:24.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:24.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246519071999998, 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:24.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246485329999999, 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:24.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246438292000001, 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246380451999997, 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:24.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:24.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246321365, 2024-09-12T09:30:24.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:24.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246287178000003, 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:24.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:24.918 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246255841999997, 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:24.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:24.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246225801000001, 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:24.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:24.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246194555999999, 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:24.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:24.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:24.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246152787, 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246121320999997, 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:24.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:24.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246073781, 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.246040743999998, 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:24.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245995723999997, 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:24.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245963066000002, 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:24.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245931251000002, 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:24.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:24.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245873338000003, 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:24.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245831355999997, 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:24.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.245795373999997, 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:24.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:24.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:24.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:24.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:24.978 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:24.980 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:24.980 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:24.976+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:24.983 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:25.000 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:25.000 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:25.018 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:25.038 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:25.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:25.060 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:25.079 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:25.072+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:25.081 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:25.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:25.103 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:25.152 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.152 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:25.212 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:25.212+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:25.281 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:25.280+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:25.289 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.289 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:25.382 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:25.380+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:25.461 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.461 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:25.520 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:25.516+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:25.584 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:25.584+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:25.610 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.610 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:25.712 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:25.708+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:25.785 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:25.784+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:25.789 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.789 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:25.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:30:25.887 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:25.884+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:25.968 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:25.968 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:25.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:25.987 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:25.984+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:26.087 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:26.080+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:26.101 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:26.101 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:26.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:26.204 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:26.200+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:26.283 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:26.283 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:26.290 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:26.284+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:26.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:26.391 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:26.388+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:26.472 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:26.472 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:26.475 INFO:teuthology.orchestra.run.smithi007.stdout:1575 writesame 0x2428673 thru 0x2431d42 data_size 0x7f(0x96d0 bytes) 2024-09-12T09:30:26.479 INFO:teuthology.orchestra.run.smithi007.stdout:1577 read 0x81c80fd thru 0x81c977f (0x1683 bytes) 2024-09-12T09:30:26.480 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.480 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:26.492 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:26.488+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:26.506 INFO:teuthology.orchestra.run.smithi007.stdout:1582 write 0xcc763a1 thru 0xcc7a423 (0x4083 bytes) 2024-09-12T09:30:26.509 INFO:teuthology.orchestra.run.smithi007.stdout:1584 read 0x85b5dac thru 0x85b9b96 (0x3deb bytes) 2024-09-12T09:30:26.510 INFO:teuthology.orchestra.run.smithi007.stdout:1585 read 0x6fbb1c5 thru 0x6fc38b3 (0x86ef bytes) 2024-09-12T09:30:26.510 INFO:teuthology.orchestra.run.smithi007.stdout:1586 read 0x747ac50 thru 0x74839bc (0x8d6d bytes) 2024-09-12T09:30:26.512 INFO:teuthology.orchestra.run.smithi007.stdout:1587 read 0x2b0e1fd thru 0x2b17ed3 (0x9cd7 bytes) 2024-09-12T09:30:26.512 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.512 INFO:teuthology.orchestra.run.smithi007.stdout:1589 writesame 0xc999a3a thru 0xc9a1b7e data_size 0x3(0x8145 bytes) 2024-09-12T09:30:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:1592 write 0xa19f47e thru 0xa1ac7d0 (0xd353 bytes) 2024-09-12T09:30:26.536 INFO:teuthology.orchestra.run.smithi007.stdout:1593 writesame 0xeb6a2c1 thru 0xeb7225b data_size 0x1(0x7f9b bytes) 2024-09-12T09:30:26.537 INFO:teuthology.orchestra.run.smithi007.stdout:1594 write 0xb74c408 thru 0xb757b3a (0xb733 bytes) 2024-09-12T09:30:26.539 INFO:teuthology.orchestra.run.smithi007.stdout:1597 write 0x52e5fc6 thru 0x52ee83d (0x8878 bytes) 2024-09-12T09:30:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:1599 read 0xc399c20 thru 0xc39e5ac (0x498d bytes) 2024-09-12T09:30:26.542 INFO:teuthology.orchestra.run.smithi007.stdout:1600 read 0xfcc124 thru 0xfd3065 (0x6f42 bytes) 2024-09-12T09:30:26.543 INFO:teuthology.orchestra.run.smithi007.stdout:1602 write 0x5c597aa thru 0x5c5bfe6 (0x283d bytes) 2024-09-12T09:30:26.546 INFO:teuthology.orchestra.run.smithi007.stdout:1603 write 0xb87ac7d thru 0xb883c96 (0x901a bytes) 2024-09-12T09:30:26.552 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.553 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.553 INFO:teuthology.orchestra.run.smithi007.stdout:1609 write 0x3e201fe thru 0x3e27c0b (0x7a0e bytes) 2024-09-12T09:30:26.556 INFO:teuthology.orchestra.run.smithi007.stdout:1610 read 0x23a31bf thru 0x23ad7d5 (0xa617 bytes) 2024-09-12T09:30:26.556 INFO:teuthology.orchestra.run.smithi007.stdout:1611 write 0x5e1d65e thru 0x5e23dd8 (0x677b bytes) 2024-09-12T09:30:26.577 INFO:teuthology.orchestra.run.smithi007.stdout:1612 trunc from 0xeb7225c to 0x45da45a 2024-09-12T09:30:26.593 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:26.592+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:26.600 INFO:teuthology.orchestra.run.smithi007.stdout:1615 read 0x1fd7caf thru 0x1fe46c6 (0xca18 bytes) 2024-09-12T09:30:26.606 INFO:teuthology.orchestra.run.smithi007.stdout:1616 trunc from 0x45da45a to 0x1195083 2024-09-12T09:30:26.627 INFO:teuthology.orchestra.run.smithi007.stdout:1620 read 0xc2fa64 thru 0xc34aff (0x509c bytes) 2024-09-12T09:30:26.627 INFO:teuthology.orchestra.run.smithi007.stdout:1621 read 0xb3f81f thru 0xb46abc (0x729e bytes) 2024-09-12T09:30:26.647 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:26.647 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:26.653 INFO:teuthology.orchestra.run.smithi007.stdout:1622 write 0x601b241 thru 0x60299dd (0xe79d bytes) 2024-09-12T09:30:26.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:26.681 INFO:teuthology.orchestra.run.smithi007.stdout:1623 writesame 0xc30b91e thru 0xc31201d data_size 0x10(0x6700 bytes) 2024-09-12T09:30:26.683 INFO:teuthology.orchestra.run.smithi007.stdout:1625 read 0x10fccf4 thru 0x1102061 (0x536e bytes) 2024-09-12T09:30:26.683 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.683 INFO:teuthology.orchestra.run.smithi007.stdout:1628 read 0x792fe52 thru 0x793ce05 (0xcfb4 bytes) 2024-09-12T09:30:26.684 INFO:teuthology.orchestra.run.smithi007.stdout:1630 write 0x46a8925 thru 0x46af9c9 (0x70a5 bytes) 2024-09-12T09:30:26.686 INFO:teuthology.orchestra.run.smithi007.stdout:1631 write 0x1aab42e thru 0x1ab3460 (0x8033 bytes) 2024-09-12T09:30:26.688 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:26.688 INFO:teuthology.orchestra.run.smithi007.stdout:1634 read 0x759a80d thru 0x759e8a9 (0x409d bytes) 2024-09-12T09:30:26.688 INFO:teuthology.orchestra.run.smithi007.stdout:1635 write 0x97d61cf thru 0x97d6264 (0x96 bytes) 2024-09-12T09:30:26.691 INFO:teuthology.orchestra.run.smithi007.stdout:1636 write 0x5f8ddc9 thru 0x5f92807 (0x4a3f bytes) 2024-09-12T09:30:26.694 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:26.692+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:26.695 INFO:teuthology.orchestra.run.smithi007.stdout:1638 write 0x9776efd thru 0x9784eef (0xdff3 bytes) 2024-09-12T09:30:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:1639 trunc from 0xc31201e to 0x3ab3339 2024-09-12T09:30:26.731 INFO:teuthology.orchestra.run.smithi007.stdout:1640 trunc from 0x3ab3339 to 0x1bed303 2024-09-12T09:30:26.747 INFO:teuthology.orchestra.run.smithi007.stdout:1642 trunc from 0x1bed303 to 0x1fd92b3 2024-09-12T09:30:26.762 INFO:teuthology.orchestra.run.smithi007.stdout:1643 clone 18 order 22 su 16384 sc 2 2024-09-12T09:30:26.794 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:26.792+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:26.818 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:26.818 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:30:26.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:26.897 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:26.892+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:26.996 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:26.992+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:27.023 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:27.024 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:27.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:27.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:27.124 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:27.120+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:27.197 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:27.192+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:27.207 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:27.207 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:27.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:27.273 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:27.292 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:27.298 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:27.296+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:27.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:27.325 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:27.346 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:27.367 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:27.367 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:27.380 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:27.381 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:27.388 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:27.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:27.399 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:27.396+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:27.409 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:27.500 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:27.500+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:27.535 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:27.535 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:27.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53986 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 14909440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.146966415000001, 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055388721000000002, 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53986 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:27.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387067+0000", 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387069+0000", 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.905e-06 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387076+0000", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387087+0000", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1499e-05 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442148+0000", 2024-09-12T09:30:27.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055060410999999997 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442229+0000", 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1441999999999997e-05 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442456+0000", 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000226534 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53987 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15007744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.146938012, 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055709171000000002, 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53987 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387095+0000", 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387096+0000", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387098+0000", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0770000000000002e-06 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387105+0000", 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7619999999999998e-06 2024-09-12T09:30:27.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387113+0000", 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4959999999999996e-06 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442489+0000", 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055376140999999997 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442610+0000", 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120234 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442805+0000", 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019502899999999999 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.545 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:53988 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 15106048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.146912619999998, 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057056229, 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53988 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387121+0000", 2024-09-12T09:30:27.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387123+0000", 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387130+0000", 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5810000000000002e-06 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387135+0000", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1490000000000002e-06 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442836+0000", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055700889000000003 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.442918+0000", 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000004e-05 2024-09-12T09:30:27.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.444177+0000", 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012590570000000001 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54006 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17367040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.146706202000001, 2024-09-12T09:30:27.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056089721000000002, 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54006 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:27.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.387327+0000", 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389586+0000", 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002258552 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389632+0000", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5796e-05 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389644+0000", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2699e-05 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.389667+0000", 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3076999999999999e-05 2024-09-12T09:30:27.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443194+0000", 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053526642999999999 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443320+0000", 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012563599999999999 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.443417+0000", 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7318000000000006e-05 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54034 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 17694720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.142943437, 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055444460000000001, 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54034 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391090+0000", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391755+0000", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066475499999999995 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391760+0000", 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.181e-06 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391904+0000", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014413300000000001 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.391916+0000", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1507e-05 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446235+0000", 2024-09-12T09:30:27.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054319522000000002 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446313+0000", 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7745999999999997e-05 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.446534+0000", 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221616 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54051 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20250624~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.141272449999999, 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055664077999999999, 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54051 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392761+0000", 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392762+0000", 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392764+0000", 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4710000000000001e-06 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392772+0000", 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6969999999999993e-06 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392777+0000", 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.587e-06 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447587+0000", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.05480952 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.447661+0000", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4635000000000006e-05 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448425+0000", 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076352799999999997 2024-09-12T09:30:27.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:54052 3.b 3:d7e7962e:::rbd_data.119172280d2a.0000000000000006:c [read 20348928~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.141245729000001, 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056119461000000002, 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54052 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.392788+0000", 2024-09-12T09:30:27.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393045+0000", 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025747600000000003 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393054+0000", 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0159999999999997e-06 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393064+0000", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4050000000000006e-06 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.393071+0000", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6290000000000001e-06 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448481+0000", 2024-09-12T09:30:27.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055409753999999999 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448576+0000", 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4969999999999994e-05 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.448907+0000", 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033121099999999998 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61227 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13533184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293779931, 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055556323999999997, 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61227 2024-09-12T09:30:27.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240253+0000", 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240254+0000", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2600000000000002e-07 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240257+0000", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.762e-06 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240265+0000", 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8469999999999996e-06 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240271+0000", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0689999999999998e-06 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295548+0000", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055277706000000003 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295630+0000", 2024-09-12T09:30:27.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1450000000000001e-05 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295810+0000", 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017996399999999999 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61229 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13631488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293753354, 2024-09-12T09:30:27.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055868963000000001, 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61229 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240280+0000", 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240281+0000", 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240283+0000", 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240290+0000", 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0570000000000004e-06 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240295+0000", 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.164e-06 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295854+0000", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055558626999999999 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.295952+0000", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8405000000000004e-05 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296149+0000", 2024-09-12T09:30:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019671099999999999 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61231 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13729792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293729253, 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056138067999999999, 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61231 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240304+0000", 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240305+0000", 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240307+0000", 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.294e-06 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240315+0000", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1019999999999997e-06 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240321+0000", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9370000000000001e-06 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296180+0000", 2024-09-12T09:30:27.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055859345999999997 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296248+0000", 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7596999999999997e-05 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296442+0000", 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019433699999999999 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61233 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13828096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293704487999999, 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056534344, 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61233 2024-09-12T09:30:27.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240329+0000", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240332+0000", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240340+0000", 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-12T09:30:27.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240346+0000", 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296492+0000", 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056146433000000003 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296606+0000", 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011335799999999999 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296863+0000", 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000257612 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61235 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 13926400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293677972999999, 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056785587999999998, 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61235 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240355+0000", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240356+0000", 2024-09-12T09:30:27.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240358+0000", 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5390000000000002e-06 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240367+0000", 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6750000000000008e-06 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240372+0000", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2789999999999998e-06 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296882+0000", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056509707999999999 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.296969+0000", 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7366000000000003e-05 2024-09-12T09:30:27.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297141+0000", 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017161100000000001 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61237 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14024704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293653521, 2024-09-12T09:30:27.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057667507, 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61237 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240380+0000", 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9400000000000001e-07 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240382+0000", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1900000000000002e-06 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240389+0000", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4060000000000002e-06 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240394+0000", 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9300000000000002e-06 2024-09-12T09:30:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297172+0000", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056777785999999997 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.297313+0000", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014155099999999999 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298047+0000", 2024-09-12T09:30:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073424999999999999 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61239 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293631379000001, 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057981773, 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61239 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240402+0000", 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9000000000000002e-07 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240405+0000", 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4389999999999999e-06 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240412+0000", 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.469e-06 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240427+0000", 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5102e-05 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298080+0000", 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057653049999999997 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298166+0000", 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5296000000000005e-05 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298384+0000", 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000218027 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61241 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14221312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293595377999999, 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058244112000000001, 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61241 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240438+0000", 2024-09-12T09:30:27.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240441+0000", 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7020000000000001e-06 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240450+0000", 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240455+0000", 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298403+0000", 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.057947945000000001 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298473+0000", 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9863999999999994e-05 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298682+0000", 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020926199999999999 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61243 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14319616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293570376, 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058591126, 2024-09-12T09:30:27.607 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61243 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240463+0000", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240466+0000", 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7010000000000001e-06 2024-09-12T09:30:27.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240473+0000", 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240479+0000", 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.327e-06 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298722+0000", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058243504000000001 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.298783+0000", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1137999999999996e-05 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299054+0000", 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027089799999999999 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61245 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14417920~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293546602999999, 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058955593000000001, 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61245 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240487+0000", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240494+0000", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5420000000000002e-06 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240511+0000", 2024-09-12T09:30:27.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7240000000000001e-05 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240518+0000", 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5989999999999998e-06 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299110+0000", 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058592712999999998 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299190+0000", 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9622000000000004e-05 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299442+0000", 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025237399999999998 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61247 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14516224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.616 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293505929, 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059307199999999997, 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61247 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240527+0000", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240528+0000", 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240531+0000", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.615e-06 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240539+0000", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6570000000000003e-06 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240545+0000", 2024-09-12T09:30:27.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0310000000000004e-06 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299503+0000", 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058957756 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299625+0000", 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012207099999999999 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299835+0000", 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209605 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:61249 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:c [read 14614528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.293479643, 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059607902999999997, 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 61249 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240554+0000", 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240557+0000", 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5780000000000001e-06 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240565+0000", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5040000000000002e-06 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.240570+0000", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0690000000000001e-06 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299884+0000", 2024-09-12T09:30:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.059313728000000003 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.299967+0000", 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3404000000000003e-05 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:13.300162+0000", 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019417999999999999 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116937 3.1e 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22365653~8043051] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.202226+0000", 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3318070979999996, 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096599979000000002, 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116937 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.202226+0000", 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.202226+0000", 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.202230+0000", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1969999999999998e-06 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.216078+0000", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013847877999999999 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.216182+0000", 2024-09-12T09:30:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010413500000000001 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.216224+0000", 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1980999999999997e-05 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.216342+0000", 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011707800000000001 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.216555+0000", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000213604 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218325+0000", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017698970000000001 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.274451+0000", 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056125909000000002 2024-09-12T09:30:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298670+0000", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024218600999999999 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298745+0000", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5717999999999996e-05 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298826+0000", 2024-09-12T09:30:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0981000000000003e-05 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:27.632 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:27.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:27.634 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:27.632+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:27.735 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:27.732+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:27.766 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:27.766 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:27.766 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:27.766 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:27.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:27.835 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:27.832+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:27.905 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:27.905 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:27.905 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:27.905 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:27.905 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:27.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:27.964 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:27.964+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:28.038 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:28.032+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:28.051 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone17 from 0x1fd92b3 (overlap 0x805cd7) to 0x335856 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:28.096 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.805210003999999, 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:28.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.345713775, 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:28.101 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:28.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:28.106 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.972296824000001, 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:28.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.972249576999999, 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:28.111 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:28.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:28.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.971956500999999, 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:28.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.971394568999999, 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:28.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.971343206, 2024-09-12T09:30:28.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:28.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:28.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.970770326, 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:28.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:28.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.970618027, 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:28.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:28.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.970260937000001, 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:28.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:28.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.969635967, 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:28.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:28.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.969584631, 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.154 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.968789841, 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.968714951000001, 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:28.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.968662631000001, 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:28.164 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:28.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:28.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.968614033, 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:28.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.968359568, 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:28.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:28.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:28.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:28.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58810 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14483456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.964957303, 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069618174000000005, 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58810 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129306+0000", 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129311+0000", 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129327+0000", 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6446000000000001e-05 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129338+0000", 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0865e-05 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198581+0000", 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069242608999999997 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198675+0000", 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4285999999999998e-05 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198923+0000", 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024778499999999999 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.964905278, 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:28.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.907336731, 2024-09-12T09:30:28.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:28.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:30:28.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.195 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:28.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:28.197 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:28.192+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:28.298 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:28.296+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:28.330 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:28.330 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:28.330 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:28.330 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:28.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:28.383 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-12T09:30:28.399 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:28.392+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:28.501 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:28.496+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:28.517 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:28.517 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:28.517 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:28.517 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:28.518 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:28.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:28.601 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:28.596+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.254864670000003, 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:28.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:28.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:28.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.254769408000001, 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:28.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.251084784, 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:28.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.249279694000002, 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:28.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.248733018999999, 2024-09-12T09:30:28.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:28.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.247964576000001, 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.247554207999997, 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.247415031999999, 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.239636220999998, 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.237272206999997, 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.226520313999998, 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.226323923000002, 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.224829503000002, 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.524586641999999, 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.524564224000001, 2024-09-12T09:30:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.524541083000001, 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.524516124, 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4811646100000004, 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:30:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:30:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.8646865679999998, 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:30:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.8645872780000001, 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:30:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:30:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:28.771 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:28.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:28.773 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:28.772+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:28.782 INFO:teuthology.orchestra.run.smithi007.stdout:1644 punch from 0x950cd1 to 0x957c5d, (0x6f8c bytes) 2024-09-12T09:30:28.791 INFO:teuthology.orchestra.run.smithi007.stdout:1647 read 0x1cf019b thru 0x1cf99b1 (0x9817 bytes) 2024-09-12T09:30:28.793 INFO:teuthology.orchestra.run.smithi007.stdout:1649 punch from 0xa252ca to 0xa264c7, (0x11fd bytes) 2024-09-12T09:30:28.793 INFO:teuthology.orchestra.run.smithi007.stdout:1651 read 0xbea618 thru 0xbf7e76 (0xd85f bytes) 2024-09-12T09:30:28.811 INFO:teuthology.orchestra.run.smithi007.stdout:1652 trunc from 0x1fd92b3 to 0xcd1749b 2024-09-12T09:30:28.832 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [6, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:28.832 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:28.832 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-12T09:30:28.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-12T09:30:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:1653 write 0xd2aea12 thru 0xd2bd1b2 (0xe7a1 bytes) 2024-09-12T09:30:28.843 INFO:teuthology.orchestra.run.smithi007.stdout:1655 punch from 0x46c9ba8 to 0x46d7ef4, (0xe34c bytes) 2024-09-12T09:30:28.845 INFO:teuthology.orchestra.run.smithi007.stdout:1656 trunc from 0xd2bd1b3 to 0xc2770c7 2024-09-12T09:30:28.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:30:28.863 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:28.863 INFO:teuthology.orchestra.run.smithi007.stdout:1658 writesame 0x50b3115 thru 0x50baa9f data_size 0x31(0x798b bytes) 2024-09-12T09:30:28.873 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:28.868+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:28.886 INFO:teuthology.orchestra.run.smithi007.stdout:1659 write 0xd254764 thru 0xd25600b (0x18a8 bytes) 2024-09-12T09:30:28.892 INFO:teuthology.orchestra.run.smithi007.stdout:1661 read 0x647f6cd thru 0x6480894 (0x11c8 bytes) 2024-09-12T09:30:28.892 INFO:teuthology.orchestra.run.smithi007.stdout:1662 write 0x828c492 thru 0x8290279 (0x3de8 bytes) 2024-09-12T09:30:28.904 INFO:teuthology.orchestra.run.smithi007.stdout:1663 write 0x6af4c09 thru 0x6afaf24 (0x631c bytes) 2024-09-12T09:30:28.911 INFO:teuthology.orchestra.run.smithi007.stdout:1665 punch from 0x846e03d to 0x8476c72, (0x8c35 bytes) 2024-09-12T09:30:28.911 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:28.911 INFO:teuthology.orchestra.run.smithi007.stdout:1669 write 0xa0f0436 thru 0xa0fc89e (0xc469 bytes) 2024-09-12T09:30:28.924 INFO:teuthology.orchestra.run.smithi007.stdout:1670 trunc from 0xd25600c to 0xa0e0141 2024-09-12T09:30:28.950 INFO:teuthology.orchestra.run.smithi007.stdout:1673 trunc from 0xa0e0141 to 0xd56c798 2024-09-12T09:30:28.955 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:28.955 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:28.955 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:28.955 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:28.961 INFO:teuthology.orchestra.run.smithi007.stdout:1674 read 0x546e375 thru 0x547291e (0x45aa bytes) 2024-09-12T09:30:28.962 INFO:teuthology.orchestra.run.smithi007.stdout:1676 write 0x9d4c2e1 thru 0x9d4da52 (0x1772 bytes) 2024-09-12T09:30:28.967 INFO:teuthology.orchestra.run.smithi007.stdout:1677 read 0x9425c56 thru 0x9427631 (0x19dc bytes) 2024-09-12T09:30:28.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:28.982 INFO:teuthology.orchestra.run.smithi007.stdout:1679 trunc from 0xd56c798 to 0x2f3bc19 2024-09-12T09:30:28.998 INFO:teuthology.orchestra.run.smithi007.stdout:1681 read 0x2ae0f90 thru 0x2ae1413 (0x484 bytes) 2024-09-12T09:30:28.998 INFO:teuthology.orchestra.run.smithi007.stdout:1682 writesame 0x2cd20d2 thru 0x2cd6758 data_size 0x17(0x4687 bytes) 2024-09-12T09:30:29.005 INFO:teuthology.orchestra.run.smithi007.stdout:1685 trunc from 0x2f3bc19 to 0x1617f30 2024-09-12T09:30:29.016 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:29.016+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:29.068 INFO:teuthology.orchestra.run.smithi007.stdout:1686 read 0xa02fce thru 0xa073f2 (0x4425 bytes) 2024-09-12T09:30:29.069 INFO:teuthology.orchestra.run.smithi007.stdout:1687 punch from 0x19ecd4 to 0x1a20d3, (0x33ff bytes) 2024-09-12T09:30:29.069 INFO:teuthology.orchestra.run.smithi007.stdout:1688 punch from 0xbc7eaa to 0xbcb84f, (0x39a5 bytes) 2024-09-12T09:30:29.069 INFO:teuthology.orchestra.run.smithi007.stdout:1689 writesame 0xca2bd2 thru 0xcb2796 data_size 0x1(0xfbc5 bytes) 2024-09-12T09:30:29.086 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:29.068+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:29.111 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:29.112 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:29.128 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:29.178 INFO:teuthology.orchestra.run.smithi007.stdout:1690 writesame 0xa420382 thru 0xa42eae3 data_size 0x7(0xe762 bytes) 2024-09-12T09:30:29.196 INFO:teuthology.orchestra.run.smithi007.stdout:1691 writesame 0xeccdf26 thru 0xecce65d data_size 0x18(0x738 bytes) 2024-09-12T09:30:29.196 INFO:teuthology.orchestra.run.smithi007.stdout:1693 writesame 0x4217b9d thru 0x421dec1 data_size 0x1(0x6325 bytes) 2024-09-12T09:30:29.216 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:29.216+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:29.235 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:30:29.228+0000 7f3454e55640 -1 mon.a@0(leader).osd e68 definitely_dead 0 2024-09-12T09:30:29.236 INFO:teuthology.orchestra.run.smithi007.stdout:1694 trunc from 0xecce65e to 0x387a128 2024-09-12T09:30:29.263 INFO:teuthology.orchestra.run.smithi007.stdout:1697 trunc from 0x387a128 to 0xba33ebb 2024-09-12T09:30:29.273 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:29.273 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:29.273 INFO:teuthology.orchestra.run.smithi007.stdout:1700 punch from 0x8a23bfa to 0x8a2cd48, (0x914e bytes) 2024-09-12T09:30:29.273 INFO:teuthology.orchestra.run.smithi007.stdout:1701 write 0x363e386 thru 0x3645b46 (0x77c1 bytes) 2024-09-12T09:30:29.278 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:29.276+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:29.280 INFO:teuthology.orchestra.run.smithi007.stdout:1702 read 0x61327f2 thru 0x613e726 (0xbf35 bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1703 punch from 0x183c9f0 to 0x183cc72, (0x282 bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1704 read 0x603075f thru 0x60386dd (0x7f7f bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1705 punch from 0x15ce4fe to 0x15d3d01, (0x5803 bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1706 read 0xadeb004 thru 0xadf1fa8 (0x6fa5 bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1707 read 0x78c96c5 thru 0x78d1a16 (0x8352 bytes) 2024-09-12T09:30:29.281 INFO:teuthology.orchestra.run.smithi007.stdout:1709 clone 19 order 18 su 8192 sc 11 2024-09-12T09:30:29.307 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669953216000003, 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669903607999998, 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669869865999999, 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669822828000001, 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669764987999997, 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669705901, 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669671714000003, 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669640377999997, 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669610337000002, 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669579091999999, 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669537323, 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669505856999997, 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669458317, 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669425279999999, 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669380259999997, 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:29.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:29.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:29.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669347602000002, 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:29.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669315787000002, 2024-09-12T09:30:29.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:29.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:29.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669257874000003, 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:29.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669215891999997, 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.669179909999997, 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:29.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:29.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:29.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:29.404 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:29.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:29.406 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:29.404+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:29.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:29.507 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:29.504+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:29.574 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:29.575 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:29.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:29.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:29.590 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:29.605 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:29.607 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:29.600+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:29.622 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:29.622 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:29.639 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:29.639 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:29.660 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:29.660 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:29.679 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:29.679 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:29.697 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:29.708 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:29.708+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:29.719 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:29.750 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:29.750 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:29.765 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:29.809 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:29.808+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.7. 2024-09-12T09:30:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:30:29.885 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-12T09:30:29.934 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:29.934 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:29.936 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:29.932+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:29.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:30.010 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:30.008+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:30.075 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.075 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:30.142 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:30.136+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:30.243 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:30.236+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:30.268 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.268 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:30.344 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:30.344+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:30.445 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:30.444+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:30.452 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.452 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:30.546 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:30.540+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:30.595 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.595 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:30.652 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:30.648+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:30.748 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:30.744+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:30.781 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.781 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:30.884 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:30.880+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:30.950 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:30.944+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:30.979 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:30.979 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:30.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:31.051 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:31.048+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:31.152 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:31.152+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:31.253 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:31.252+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:31.300 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.300 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:31.354 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:31.352+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:31.433 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.433 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:31.496 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:31.492+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:31.555 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:31.548+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:31.576 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.577 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:31.656 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:31.656+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:31.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:31.753 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.753 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.756 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:31.756+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:31.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:31.856 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:31.852+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:31.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:30:31.892 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:31.911 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:31.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:31.938 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:31.938 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:31.948 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:31.957 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:31.956+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:31.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:31.972 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:31.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:32.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:32.017 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:32.058 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:32.056+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:32.089 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone18 from 0xba33ebb (overlap 0x1617f30) to 0x11acc26 2024-09-12T09:30:32.108 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:32.109 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:32.109 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:32.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:32.168 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:32.168+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:32.261 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:32.256+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:32.301 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:32.302 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:32.302 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:32.302 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:32.302 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:32.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:32.361 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:32.356+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:32.390 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-12T09:30:32.411 INFO:teuthology.orchestra.run.smithi007.stdout:1710 writesame 0x375363a thru 0x375c59c data_size 0x2f(0x8f63 bytes) 2024-09-12T09:30:32.461 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:32.461 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.169705560000001, 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:32.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.710209331000001, 2024-09-12T09:30:32.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:32.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:32.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:32.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.336792379999999, 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:32.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:32.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:32.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:32.476 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.336745133000001, 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.336452056999999, 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:32.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:32.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:32.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.335890124999999, 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.486 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:32.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:32.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:32.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.335838762000002, 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:32.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.335265882000002, 2024-09-12T09:30:32.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:32.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:32.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:32.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.335113582999998, 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:32.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:32.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.334756493, 2024-09-12T09:30:32.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.334131523, 2024-09-12T09:30:32.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:32.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:32.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.334080187000001, 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:32.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.333285397000001, 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:32.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.333210507, 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.531 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.333158186999999, 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.333109588999999, 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.332855124000002, 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:32.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58810 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14483456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.329452859, 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069618174000000005, 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58810 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129305+0000", 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129306+0000", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2580000000000001e-06 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129311+0000", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129327+0000", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6446000000000001e-05 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129338+0000", 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0865e-05 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198581+0000", 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069242608999999997 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198675+0000", 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4285999999999998e-05 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198923+0000", 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024778499999999999 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.329400834000001, 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:32.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2718322870000005, 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:32.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:32.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:32.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:32.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:32.561 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:32.562 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:32.456+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:32.563 INFO:teuthology.orchestra.run.smithi007.stdout:1711 punch from 0x17367d8 to 0x1739260, (0x2a88 bytes) 2024-09-12T09:30:32.563 INFO:teuthology.orchestra.run.smithi007.stdout:1713 read 0x2cbdacb thru 0x2cc5d9d (0x82d3 bytes) 2024-09-12T09:30:32.563 INFO:teuthology.orchestra.run.smithi007.stdout:1714 trunc from 0xba33ebb to 0x61d623 2024-09-12T09:30:32.564 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:32.556+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:32.564 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:32.599 INFO:teuthology.orchestra.run.smithi007.stdout:1715 read 0x4afff thru 0x4d421 (0x2423 bytes) 2024-09-12T09:30:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:1717 trunc from 0x61d623 to 0xca2236e 2024-09-12T09:30:32.632 INFO:teuthology.orchestra.run.smithi007.stdout:1718 writesame 0x1c86d72 thru 0x1c87dab data_size 0x2(0x103a bytes) 2024-09-12T09:30:32.638 INFO:teuthology.orchestra.run.smithi007.stdout:1719 writesame 0x13b1e80 thru 0x13c0494 data_size 0x1(0xe615 bytes) 2024-09-12T09:30:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:30:32.663 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:32.660+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:32.712 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:32.713 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:32.713 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:32.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:32.772 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:32.772+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:32.866 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:32.860+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:32.909 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:32.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:32.909 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:32.910 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:32.910 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:32.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:32.967 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:32.960+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.648749295000002, 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:33.061 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:33.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:33.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.648654033, 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:33.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:33.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:33.072 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.644969408999998, 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:33.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:33.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:33.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.643164319, 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:33.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:33.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.642617643999998, 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:33.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.641849200999999, 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.089 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:33.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.641438833000002, 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.641299656999998, 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.633520846000003, 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:33.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:33.110 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.631156832000002, 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:33.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:33.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.620404938999997, 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:33.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:33.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.123 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.620208548000001, 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:33.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:33.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:33.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:33.129 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 43.618714128000001, 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.918471267000001, 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:33.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:33.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.918448849000001, 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:33.140 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:33.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:33.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.918425708000001, 2024-09-12T09:30:33.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:33.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:33.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:33.150 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.918400749, 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:33.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.8750492350000005, 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:30:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:30:33.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2585711929999999, 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:30:33.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:30:33.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.166 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2584719030000002, 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:33.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:33.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:30:33.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.172 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:33.173 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:33.174 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:33.168+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:33.275 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:33.272+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:33.304 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:33.305 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:33.305 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:33.305 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:33.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:33.404 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:33.404+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:33.476 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:33.476+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:33.483 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:33.483 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:33.483 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:33.483 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:33.483 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:33.497 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:33.577 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:33.572+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:33.665 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027717717000002, 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:33.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027668108999997, 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.675 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027634366999997, 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:33.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027587328999999, 2024-09-12T09:30:33.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:33.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027529489000003, 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:33.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:33.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027470401999999, 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:33.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:33.694 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027436215000002, 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:33.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027404879000002, 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:33.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027374838, 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:33.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:33.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027343592999998, 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:33.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:33.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:33.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027301823999998, 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:33.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027270358000003, 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027222817999998, 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:33.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:33.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027189780999997, 2024-09-12T09:30:33.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:33.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:33.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027144761000002, 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:33.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:33.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027112103, 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:33.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:33.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027080288000001, 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:33.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:33.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.027022375000001, 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:33.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.026980393000002, 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:33.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.026944411000002, 2024-09-12T09:30:33.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:33.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.763 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:33.764 INFO:teuthology.orchestra.run.smithi007.stdout:1720 punch from 0x6721ce to 0x67fbff, (0xda31 bytes) 2024-09-12T09:30:33.764 INFO:teuthology.orchestra.run.smithi007.stdout:1721 write 0x29284b7 thru 0x29286c7 (0x211 bytes) 2024-09-12T09:30:33.765 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:33.765 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:33.764+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:33.866 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:33.860+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:33.890 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:33.890 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:33.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122461 3.15 3.3f6b9f95 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:33.681143+0000", 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.20993086699999999, 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.20997228300000001, 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122461 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681143+0000", 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681143+0000", 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:33.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681146+0000", 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2270000000000002e-06 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681157+0000", 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1212999999999999e-05 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681189+0000", 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1353000000000003e-05 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681211+0000", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2302000000000001e-05 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681375+0000", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016344999999999999 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.681394+0000", 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9502999999999999e-05 2024-09-12T09:30:33.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:30:33.895 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:33.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:33.967 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:33.964+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:34.028 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:34.029 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:34.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:34.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:34.068 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:34.064+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:34.168 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:34.164+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:34.215 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815806795999997, 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:30:34.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815747385999998, 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:34.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815690799999999, 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:34.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815622486999999, 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:34.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815564301000002, 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:34.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:34.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815503767999999, 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:34.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:34.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815443768000002, 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:34.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815387418999997, 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:34.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815328108999999, 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:34.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815272475999997, 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:34.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815216092999997, 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:34.263 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:34.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:34.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815159522999998, 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:34.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:34.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815103106999999, 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:34.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:34.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:34.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.815047614000001, 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.814959600999998, 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.814895157999999, 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.968447372, 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.785031187000001, 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.668965084, 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.555935225999999, 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:34.315 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:34.318 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:34.316+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:34.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:34.341 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:34.432 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:34.428+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:34.456 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:34.457 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:34.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:34.556 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:34.552+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:34.620 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:34.620+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:34.641 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:34.641 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:34.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:34.676 INFO:teuthology.orchestra.run.smithi007.stdout:1722 read 0x1168c4 thru 0x12388f (0xcfcc bytes) 2024-09-12T09:30:34.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:30:34.680 INFO:teuthology.orchestra.run.smithi007.stdout:1723 read 0xbd037ba thru 0xbd1351a (0xfd61 bytes) 2024-09-12T09:30:34.681 INFO:teuthology.orchestra.run.smithi007.stdout:1724 punch from 0x9de6ed2 to 0x9deca29, (0x5b57 bytes) 2024-09-12T09:30:34.681 INFO:teuthology.orchestra.run.smithi007.stdout:1725 write 0x813b962 thru 0x814064e (0x4ced bytes) 2024-09-12T09:30:34.699 INFO:teuthology.orchestra.run.smithi007.stdout:1727 write 0x5b3ff27 thru 0x5b454c1 (0x559b bytes) 2024-09-12T09:30:34.714 INFO:teuthology.orchestra.run.smithi007.stdout:1729 write 0xbe26228 thru 0xbe2b1ff (0x4fd8 bytes) 2024-09-12T09:30:34.720 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:34.720+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:34.725 INFO:teuthology.orchestra.run.smithi007.stdout:1730 read 0xef6a47 thru 0xf00072 (0x962c bytes) 2024-09-12T09:30:34.726 INFO:teuthology.orchestra.run.smithi007.stdout:1731 write 0xaa44028 thru 0xaa4f23c (0xb215 bytes) 2024-09-12T09:30:34.743 INFO:teuthology.orchestra.run.smithi007.stdout:1732 read 0x3c1cddd thru 0x3c208ad (0x3ad1 bytes) 2024-09-12T09:30:34.743 INFO:teuthology.orchestra.run.smithi007.stdout:1733 write 0xa87d931 thru 0xa88bf71 (0xe641 bytes) 2024-09-12T09:30:34.759 INFO:teuthology.orchestra.run.smithi007.stdout:1736 write 0x3381a18 thru 0x338b35c (0x9945 bytes) 2024-09-12T09:30:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:1737 punch from 0xab8e3dc to 0xab8feb1, (0x1ad5 bytes) 2024-09-12T09:30:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:1741 read 0x47ff524 thru 0x4805cf0 (0x67cd bytes) 2024-09-12T09:30:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:1742 read 0x8c4c435 thru 0x8c50515 (0x40e1 bytes) 2024-09-12T09:30:34.779 INFO:teuthology.orchestra.run.smithi007.stdout:1744 write 0x360c600 thru 0x361536b (0x8d6c bytes) 2024-09-12T09:30:34.790 INFO:teuthology.orchestra.run.smithi007.stdout:1745 read 0x312ef33 thru 0x3139ebd (0xaf8b bytes) 2024-09-12T09:30:34.790 INFO:teuthology.orchestra.run.smithi007.stdout:1746 writesame 0x70514ae thru 0x70594c9 data_size 0xc(0x801c bytes) 2024-09-12T09:30:34.804 INFO:teuthology.orchestra.run.smithi007.stdout:1747 punch from 0x591dff9 to 0x592869d, (0xa6a4 bytes) 2024-09-12T09:30:34.804 INFO:teuthology.orchestra.run.smithi007.stdout:1748 punch from 0x63c77f7 to 0x63d35d5, (0xbdde bytes) 2024-09-12T09:30:34.804 INFO:teuthology.orchestra.run.smithi007.stdout:1749 writesame 0x2edaf4b thru 0x2ee06ef data_size 0x1b(0x57a5 bytes) 2024-09-12T09:30:34.818 INFO:teuthology.orchestra.run.smithi007.stdout:1750 write 0xb6245a8 thru 0xb6338d2 (0xf32b bytes) 2024-09-12T09:30:34.820 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:34.820 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:34.821 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:34.816+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:34.837 INFO:teuthology.orchestra.run.smithi007.stdout:1751 read 0x155c2b4 thru 0x1561596 (0x52e3 bytes) 2024-09-12T09:30:34.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:34.850 INFO:teuthology.orchestra.run.smithi007.stdout:1752 writesame 0xdfca9e8 thru 0xdfd14e6 data_size 0x31(0x6aff bytes) 2024-09-12T09:30:34.856 INFO:teuthology.orchestra.run.smithi007.stdout:1753 trunc from 0xdfd14e7 to 0x99b4f03 2024-09-12T09:30:34.881 INFO:teuthology.orchestra.run.smithi007.stdout:1755 read 0x165d7da thru 0x16611e7 (0x3a0e bytes) 2024-09-12T09:30:34.881 INFO:teuthology.orchestra.run.smithi007.stdout:1756 read 0x848d5e7 thru 0x848e06a (0xa84 bytes) 2024-09-12T09:30:34.883 INFO:teuthology.orchestra.run.smithi007.stdout:1757 trunc from 0x99b4f03 to 0x8668a7f 2024-09-12T09:30:34.886 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:34.886 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:34.886 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:30:34.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:30:34.893 INFO:teuthology.orchestra.run.smithi007.stdout:1758 writesame 0x56a13ad thru 0x56adc4f data_size 0xd(0xc8a3 bytes) 2024-09-12T09:30:34.922 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:34.916+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:34.931 INFO:teuthology.orchestra.run.smithi007.stdout:1759 write 0xdcccab1 thru 0xdcd11ba (0x470a bytes) 2024-09-12T09:30:34.939 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:34.961 INFO:teuthology.orchestra.run.smithi007.stdout:1763 trunc from 0xdcd11bb to 0xf2e6b1 2024-09-12T09:30:35.002 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.002 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.006 INFO:teuthology.orchestra.run.smithi007.stdout:1766 writesame 0x1088c9b thru 0x1093c66 data_size 0x2(0xafcc bytes) 2024-09-12T09:30:35.022 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:35.016+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:35.026 INFO:teuthology.orchestra.run.smithi007.stdout:1767 trunc from 0x1093c67 to 0xc9e66b1 2024-09-12T09:30:35.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:35.038 INFO:teuthology.orchestra.run.smithi007.stdout:1768 writesame 0x8afbe3b thru 0x8b0bc36 data_size 0xa(0xfdfc bytes) 2024-09-12T09:30:35.061 INFO:teuthology.orchestra.run.smithi007.stdout:1769 read 0x326556e thru 0x3269b57 (0x45ea bytes) 2024-09-12T09:30:35.061 INFO:teuthology.orchestra.run.smithi007.stdout:1770 write 0x3abc1d thru 0x3acf05 (0x12e9 bytes) 2024-09-12T09:30:35.090 INFO:teuthology.orchestra.run.smithi007.stdout:1772 trunc from 0xc9e66b1 to 0x227fa89 2024-09-12T09:30:35.110 INFO:teuthology.orchestra.run.smithi007.stdout:1774 punch from 0x1f00a12 to 0x1f03329, (0x2917 bytes) 2024-09-12T09:30:35.123 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:35.120+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:35.132 INFO:teuthology.orchestra.run.smithi007.stdout:1775 writesame 0x8fd9477 thru 0x8fdad30 data_size 0xf(0x18ba bytes) 2024-09-12T09:30:35.132 INFO:teuthology.orchestra.run.smithi007.stdout:1777 punch from 0x3ac2b3d to 0x3acd720, (0xabe3 bytes) 2024-09-12T09:30:35.138 INFO:teuthology.orchestra.run.smithi007.stdout:1778 trunc from 0x8fdad31 to 0xcbcc5fe 2024-09-12T09:30:35.160 INFO:teuthology.orchestra.run.smithi007.stdout:1779 write 0xcc6cc02 thru 0xcc7b93c (0xed3b bytes) 2024-09-12T09:30:35.173 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.173 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.183 INFO:teuthology.orchestra.run.smithi007.stdout:1780 read 0xb662bb thru 0xb75bfd (0xf943 bytes) 2024-09-12T09:30:35.183 INFO:teuthology.orchestra.run.smithi007.stdout:1781 punch from 0x7fbf0df to 0x7fc78c2, (0x87e3 bytes) 2024-09-12T09:30:35.183 INFO:teuthology.orchestra.run.smithi007.stdout:1782 read 0x414a942 thru 0x414f3e6 (0x4aa5 bytes) 2024-09-12T09:30:35.184 INFO:teuthology.orchestra.run.smithi007.stdout:1784 write 0x39814d5 thru 0x398d723 (0xc24f bytes) 2024-09-12T09:30:35.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:35.205 INFO:teuthology.orchestra.run.smithi007.stdout:1785 write 0xaebe616 thru 0xaec677b (0x8166 bytes) 2024-09-12T09:30:35.222 INFO:teuthology.orchestra.run.smithi007.stdout:1786 punch from 0xb1caeea to 0xb1da8e3, (0xf9f9 bytes) 2024-09-12T09:30:35.222 INFO:teuthology.orchestra.run.smithi007.stdout:1787 write 0xb8bef02 thru 0xb8c264e (0x374d bytes) 2024-09-12T09:30:35.223 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:35.220+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:35.232 INFO:teuthology.orchestra.run.smithi007.stdout:1789 write 0x5e5faee thru 0x5e64653 (0x4b66 bytes) 2024-09-12T09:30:35.245 INFO:teuthology.orchestra.run.smithi007.stdout:1791 write 0x9c48a8f thru 0x9c4aedc (0x244e bytes) 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 74, 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:30:34.654265+0000", 2024-09-12T09:30:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:29:16.082807+0000", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:30:29.841680+0000", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5801984, 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:30:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:30:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:30:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-12T09:30:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:30:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.9900000095367432, 2024-09-12T09:30:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.9900000095367432, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.630000114440918, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.630000114440918, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:30:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:30:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "69", 2024-09-12T09:30:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 21, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 69, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:30:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:30:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.7400000095367432, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.7400000095367432, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 73, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 66, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:30:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:30:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:30:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 51, 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:30:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:30:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 72, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:30:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:30:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 72, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:30:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 72, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:30:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:30:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:30:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 73, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:30:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:30:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:30:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:30:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:30:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:30:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:35.325 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:35.326 INFO:teuthology.orchestra.run.smithi007.stdout:1792 write 0x89e725f thru 0x89ec83f (0x55e1 bytes) 2024-09-12T09:30:35.326 INFO:teuthology.orchestra.run.smithi007.stdout:1793 write 0x65986fc thru 0x65a1d65 (0x966a bytes) 2024-09-12T09:30:35.327 INFO:tasks.thrashosds.thrasher:j is {'epoch': 74, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:30:34.654265+0000', 'last_up_change': '2024-09-12T09:29:16.082807+0000', 'last_in_change': '2024-09-12T09:30:29.841680+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:30:35.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:30:35.328 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.328 INFO:teuthology.orchestra.run.smithi007.stdout:1797 writesame 0xaaea71b thru 0xaaf90d9 data_size 0x1(0xe9bf bytes) 2024-09-12T09:30:35.328 INFO:teuthology.orchestra.run.smithi007.stdout:1798 write 0x4e132c thru 0x4e7156 (0x5e2b bytes) 2024-09-12T09:30:35.329 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:35.320+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:35.333 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.333 INFO:teuthology.orchestra.run.smithi007.stdout:1800 punch from 0x893b8d4 to 0x893d5dd, (0x1d09 bytes) 2024-09-12T09:30:35.338 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.338 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:1803 write 0xd9ed2b7 thru 0xd9fa7f2 (0xd53c bytes) 2024-09-12T09:30:35.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:35.359 INFO:teuthology.orchestra.run.smithi007.stdout:1804 write 0x1d98a64 thru 0x1da687c (0xde19 bytes) 2024-09-12T09:30:35.379 INFO:teuthology.orchestra.run.smithi007.stdout:1805 punch from 0x311a507 to 0x3121eb9, (0x79b2 bytes) 2024-09-12T09:30:35.379 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.393 INFO:teuthology.orchestra.run.smithi007.stdout:1808 trunc from 0xd9fa7f3 to 0x52bddc1 2024-09-12T09:30:35.421 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.422 INFO:teuthology.orchestra.run.smithi007.stdout:1810 write 0x11fe2ba thru 0x11ff1f6 (0xf3d bytes) 2024-09-12T09:30:35.426 INFO:teuthology.orchestra.run.smithi007.stdout:1811 writesame 0x42bfba4 thru 0x42c6ac3 data_size 0x38(0x6f20 bytes) 2024-09-12T09:30:35.427 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:35.424+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:35.438 INFO:teuthology.orchestra.run.smithi007.stdout:1812 read 0x3fda6e7 thru 0x3fda848 (0x162 bytes) 2024-09-12T09:30:35.438 INFO:teuthology.orchestra.run.smithi007.stdout:1813 punch from 0x1bd50ba to 0x1bd8711, (0x3657 bytes) 2024-09-12T09:30:35.438 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.442 INFO:teuthology.orchestra.run.smithi007.stdout:1815 trunc from 0x52bddc1 to 0x2b2fe59 2024-09-12T09:30:35.461 INFO:teuthology.orchestra.run.smithi007.stdout:1816 punch from 0xbc4d06 to 0xbcec93, (0x9f8d bytes) 2024-09-12T09:30:35.487 INFO:teuthology.orchestra.run.smithi007.stdout:1817 write 0xc952419 thru 0xc95de99 (0xba81 bytes) 2024-09-12T09:30:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:1818 read 0x1e53586 thru 0x1e53961 (0x3dc bytes) 2024-09-12T09:30:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:1819 writesame 0x9fc0674 thru 0x9fc5a64 data_size 0x13(0x53f1 bytes) 2024-09-12T09:30:35.515 INFO:teuthology.orchestra.run.smithi007.stdout:1820 trunc from 0xc95de9a to 0xe1461e9 2024-09-12T09:30:35.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:1824 read 0x984cfa9 thru 0x9858f6e (0xbfc6 bytes) 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:1825 punch from 0x56613bd to 0x56627f3, (0x1436 bytes) 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:1827 punch from 0xd8f4043 to 0xd8ffc29, (0xbbe6 bytes) 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:1829 clone 20 order 19 su 2048 sc 5 2024-09-12T09:30:35.528 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:35.524+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:35.629 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:35.628+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:35.638 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.638 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:30:35.698 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-12T09:30:34.658780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1030700749,"num_objects":909,"num_object_clones":189,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":909,"num_whiteouts":398,"num_read":138028,"num_read_kb":1080917,"num_write":6304,"num_write_kb":622358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":191197186,"num_keys_recovered":85,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4703,"ondisk_log_size":4703,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":317596,"kb_used_data":61824,"kb_used_omap":10,"kb_used_meta":255605,"kb_avail":660285284,"statfs":{"total":676457349120,"available":676132130816,"internally_reserved":0,"allocated":63307776,"data_stored":855970758,"data_compressed":5369164,"data_compressed_allocated":51920896,"data_compressed_original":849346714,"omap_allocated":11158,"internal_metadata":261739626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6180489,"num_objects":18,"num_object_clones":8,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":-29,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":4724,"num_read_kb":52484,"num_write":238,"num_write_kb":9113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":34882384,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.799602"},"pg_stats":[{"pgid":"3.1f","version":"68'369","reported_seq":4081,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.862160+0000","last_change":"2024-09-12T09:29:58.167241+0000","last_active":"2024-09-12T09:30:30.862160+0000","last_peered":"2024-09-12T09:30:30.862160+0000","last_clean":"2024-09-12T09:30:30.862160+0000","last_became_active":"2024-09-12T09:29:58.165903+0000","last_became_peered":"2024-09-12T09:29:58.165903+0000","last_unstale":"2024-09-12T09:30:30.862160+0000","last_undegraded":"2024-09-12T09:30:30.862160+0000","last_fullsized":"2024-09-12T09:30:30.862160+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:04.697026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":72876293,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":4845,"num_read_kb":83578,"num_write":540,"num_write_kb":35890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":4329673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'121","reported_seq":2506,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.865768+0000","last_change":"2024-09-12T09:30:01.194793+0000","last_active":"2024-09-12T09:30:30.865768+0000","last_peered":"2024-09-12T09:30:30.865768+0000","last_clean":"2024-09-12T09:30:30.865768+0000","last_became_active":"2024-09-12T09:30:01.169430+0000","last_became_peered":"2024-09-12T09:30:01.169430+0000","last_unstale":"2024-09-12T09:30:30.865768+0000","last_undegraded":"2024-09-12T09:30:30.865768+0000","last_fullsized":"2024-09-12T09:30:30.865768+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:32.228732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":36142742,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":2587,"num_read_kb":25346,"num_write":150,"num_write_kb":29742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":352930,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"68'111","reported_seq":2983,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.869798+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:30:30.869798+0000","last_peered":"2024-09-12T09:30:30.869798+0000","last_clean":"2024-09-12T09:30:30.869798+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:30:30.869798+0000","last_undegraded":"2024-09-12T09:30:30.869798+0000","last_fullsized":"2024-09-12T09:30:30.869798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:38.907052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":42981197,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":2798,"num_read_kb":64344,"num_write":101,"num_write_kb":35581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"68'118","reported_seq":3194,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.867712+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:30:30.867712+0000","last_peered":"2024-09-12T09:30:30.867712+0000","last_clean":"2024-09-12T09:30:30.867712+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:30:30.867712+0000","last_undegraded":"2024-09-12T09:30:30.867712+0000","last_fullsized":"2024-09-12T09:30:30.867712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:35.685780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":19732605,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":15,"num_read":3217,"num_read_kb":32732,"num_write":133,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"68'139","reported_seq":4298,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.860690+0000","last_change":"2024-09-12T09:29:55.231536+0000","last_active":"2024-09-12T09:30:30.860690+0000","last_peered":"2024-09-12T09:30:30.860690+0000","last_clean":"2024-09-12T09:30:30.860690+0000","last_became_active":"2024-09-12T09:29:55.137238+0000","last_became_peered":"2024-09-12T09:29:55.137238+0000","last_unstale":"2024-09-12T09:30:30.860690+0000","last_undegraded":"2024-09-12T09:30:30.860690+0000","last_fullsized":"2024-09-12T09:30:30.860690+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:15.090232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":15084041,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":4589,"num_read_kb":11152,"num_write":147,"num_write_kb":2683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1206936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"71'136","reported_seq":1396,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:32.053173+0000","last_change":"2024-09-12T09:29:36.673515+0000","last_active":"2024-09-12T09:30:32.053173+0000","last_peered":"2024-09-12T09:30:32.053173+0000","last_clean":"2024-09-12T09:30:32.053173+0000","last_became_active":"2024-09-12T09:29:34.665723+0000","last_became_peered":"2024-09-12T09:29:34.665723+0000","last_unstale":"2024-09-12T09:30:32.053173+0000","last_undegraded":"2024-09-12T09:30:32.053173+0000","last_fullsized":"2024-09-12T09:30:32.053173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:57:37.938156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":8311296,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":1406,"num_read_kb":7701,"num_write":143,"num_write_kb":29117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"61'62","reported_seq":395,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.869977+0000","last_change":"2024-09-12T09:29:36.680185+0000","last_active":"2024-09-12T09:30:30.869977+0000","last_peered":"2024-09-12T09:30:30.869977+0000","last_clean":"2024-09-12T09:30:30.869977+0000","last_became_active":"2024-09-12T09:29:34.667496+0000","last_became_peered":"2024-09-12T09:29:34.667496+0000","last_unstale":"2024-09-12T09:30:30.869977+0000","last_undegraded":"2024-09-12T09:30:30.869977+0000","last_fullsized":"2024-09-12T09:30:30.869977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:19:06.872745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":13597632,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":462,"num_read_kb":3294,"num_write":49,"num_write_kb":1030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'74","reported_seq":659,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.864183+0000","last_change":"2024-09-12T09:29:55.177304+0000","last_active":"2024-09-12T09:30:30.864183+0000","last_peered":"2024-09-12T09:30:30.864183+0000","last_clean":"2024-09-12T09:30:30.864183+0000","last_became_active":"2024-09-12T09:29:55.134985+0000","last_became_peered":"2024-09-12T09:29:55.134985+0000","last_unstale":"2024-09-12T09:30:30.864183+0000","last_undegraded":"2024-09-12T09:30:30.864183+0000","last_fullsized":"2024-09-12T09:30:30.864183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:53.709818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":8313761,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":15,"num_read":443,"num_read_kb":4565,"num_write":63,"num_write_kb":4210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":524806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"68'92","reported_seq":1832,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.862039+0000","last_change":"2024-09-12T09:29:55.167682+0000","last_active":"2024-09-12T09:30:30.862039+0000","last_peered":"2024-09-12T09:30:30.862039+0000","last_clean":"2024-09-12T09:30:30.862039+0000","last_became_active":"2024-09-12T09:29:55.136789+0000","last_became_peered":"2024-09-12T09:29:55.136789+0000","last_unstale":"2024-09-12T09:30:30.862039+0000","last_undegraded":"2024-09-12T09:30:30.862039+0000","last_fullsized":"2024-09-12T09:30:30.862039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:14:15.137416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":34289362,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2080,"num_read_kb":40004,"num_write":199,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8413,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"68'179","reported_seq":12239,"reported_epoch":69,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:29.862331+0000","last_change":"2024-09-12T09:30:30.873189+0000","last_active":"2024-09-12T09:30:29.862331+0000","last_peered":"2024-09-12T09:30:29.088530+0000","last_clean":"2024-09-12T09:30:29.088530+0000","last_became_active":"2024-09-12T09:30:06.668824+0000","last_became_peered":"2024-09-12T09:30:06.668824+0000","last_unstale":"2024-09-12T09:30:29.862331+0000","last_undegraded":"2024-09-12T09:30:29.862331+0000","last_fullsized":"2024-09-12T09:30:29.862331+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":44059788,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":12227,"num_read_kb":60790,"num_write":264,"num_write_kb":22730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2239600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"63'169","reported_seq":4670,"reported_epoch":68,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:27.904353+0000","last_change":"2024-09-12T09:30:30.865080+0000","last_active":"2024-09-12T09:30:30.865080+0000","last_peered":"2024-09-12T09:30:27.904353+0000","last_clean":"2024-09-12T09:30:27.904353+0000","last_became_active":"2024-09-12T09:30:09.704072+0000","last_became_peered":"2024-09-12T09:30:09.704072+0000","last_unstale":"2024-09-12T09:30:27.904353+0000","last_undegraded":"2024-09-12T09:30:27.904353+0000","last_fullsized":"2024-09-12T09:30:27.904353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":66512118,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":4298,"num_read_kb":69616,"num_write":213,"num_write_kb":37067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":29632415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":7,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"68'471","reported_seq":7134,"reported_epoch":71,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-12T09:30:31.959029+0000","last_change":"2024-09-12T09:30:31.959467+0000","last_active":"2024-09-12T09:30:31.959029+0000","last_peered":"2024-09-12T09:30:31.959029+0000","last_clean":"2024-09-12T09:30:29.087461+0000","last_became_active":"2024-09-12T09:30:31.958362+0000","last_became_peered":"2024-09-12T09:30:31.958362+0000","last_unstale":"2024-09-12T09:30:31.959029+0000","last_undegraded":"2024-09-12T09:30:31.873574+0000","last_fullsized":"2024-09-12T09:30:31.869689+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":21175339,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":8249,"num_read_kb":15595,"num_write":809,"num_write_kb":21637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":693960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":["4","2"],"object_location_counts":[{"shards":"2,4","objects":35}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.940419+0000","last_change":"2024-09-12T09:30:30.871108+0000","last_active":"2024-09-12T09:30:31.940419+0000","last_peered":"2024-09-12T09:30:31.940419+0000","last_clean":"2024-09-12T09:30:31.940419+0000","last_became_active":"2024-09-12T09:30:30.870346+0000","last_became_peered":"2024-09-12T09:30:30.870346+0000","last_unstale":"2024-09-12T09:30:31.940419+0000","last_undegraded":"2024-09-12T09:30:31.940419+0000","last_fullsized":"2024-09-12T09:30:31.940419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:23.248076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"68'48","reported_seq":4009,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.860895+0000","last_change":"2024-09-12T09:29:55.208992+0000","last_active":"2024-09-12T09:30:30.860895+0000","last_peered":"2024-09-12T09:30:30.860895+0000","last_clean":"2024-09-12T09:30:30.860895+0000","last_became_active":"2024-09-12T09:29:55.138408+0000","last_became_peered":"2024-09-12T09:29:55.138408+0000","last_unstale":"2024-09-12T09:30:30.860895+0000","last_undegraded":"2024-09-12T09:30:30.860895+0000","last_fullsized":"2024-09-12T09:30:30.860895+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:28:56.799851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":19443763,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":3905,"num_read_kb":13646,"num_write":38,"num_write_kb":4245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":21350,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":124,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.870282+0000","last_change":"2024-09-12T09:30:01.176975+0000","last_active":"2024-09-12T09:30:30.870282+0000","last_peered":"2024-09-12T09:30:30.870282+0000","last_clean":"2024-09-12T09:30:30.870282+0000","last_became_active":"2024-09-12T09:30:01.176687+0000","last_became_peered":"2024-09-12T09:30:01.176687+0000","last_unstale":"2024-09-12T09:30:30.870282+0000","last_undegraded":"2024-09-12T09:30:30.870282+0000","last_fullsized":"2024-09-12T09:30:30.870282+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:50:56.303434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'126","reported_seq":1665,"reported_epoch":68,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:29.108759+0000","last_change":"2024-09-12T09:30:30.862134+0000","last_active":"2024-09-12T09:30:30.862134+0000","last_peered":"2024-09-12T09:30:29.108759+0000","last_clean":"2024-09-12T09:30:29.108759+0000","last_became_active":"2024-09-12T09:30:12.711067+0000","last_became_peered":"2024-09-12T09:30:12.711067+0000","last_unstale":"2024-09-12T09:30:29.108759+0000","last_undegraded":"2024-09-12T09:30:29.108759+0000","last_fullsized":"2024-09-12T09:30:29.108759+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":24555530,"num_objects":31,"num_object_clones":10,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":1555,"num_read_kb":11414,"num_write":184,"num_write_kb":22049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.942992+0000","last_change":"2024-09-12T09:30:30.875450+0000","last_active":"2024-09-12T09:30:31.942992+0000","last_peered":"2024-09-12T09:30:31.942992+0000","last_clean":"2024-09-12T09:30:31.942992+0000","last_became_active":"2024-09-12T09:30:30.874655+0000","last_became_peered":"2024-09-12T09:30:30.874655+0000","last_unstale":"2024-09-12T09:30:31.942992+0000","last_undegraded":"2024-09-12T09:30:31.942992+0000","last_fullsized":"2024-09-12T09:30:31.942992+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:30.720674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'86","reported_seq":3694,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.870072+0000","last_change":"2024-09-12T09:29:36.680262+0000","last_active":"2024-09-12T09:30:30.870072+0000","last_peered":"2024-09-12T09:30:30.870072+0000","last_clean":"2024-09-12T09:30:30.870072+0000","last_became_active":"2024-09-12T09:29:34.667442+0000","last_became_peered":"2024-09-12T09:29:34.667442+0000","last_unstale":"2024-09-12T09:30:30.870072+0000","last_undegraded":"2024-09-12T09:30:30.870072+0000","last_fullsized":"2024-09-12T09:30:30.870072+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:10.882997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":13039090,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":3538,"num_read_kb":14585,"num_write":165,"num_write_kb":42288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.870241+0000","last_change":"2024-09-12T09:29:55.138283+0000","last_active":"2024-09-12T09:30:30.870241+0000","last_peered":"2024-09-12T09:30:30.870241+0000","last_clean":"2024-09-12T09:30:30.870241+0000","last_became_active":"2024-09-12T09:29:55.137856+0000","last_became_peered":"2024-09-12T09:29:55.137856+0000","last_unstale":"2024-09-12T09:30:30.870241+0000","last_undegraded":"2024-09-12T09:30:30.870241+0000","last_fullsized":"2024-09-12T09:30:30.870241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:50:58.825871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"63'95","reported_seq":12854,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.872581+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:30:30.872581+0000","last_peered":"2024-09-12T09:30:30.872581+0000","last_clean":"2024-09-12T09:30:30.872581+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:30:30.872581+0000","last_undegraded":"2024-09-12T09:30:30.872581+0000","last_fullsized":"2024-09-12T09:30:30.872581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:23:14.842705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":28066020,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":18,"num_read":12916,"num_read_kb":52025,"num_write":82,"num_write_kb":7932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'185","reported_seq":3499,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.870659+0000","last_change":"2024-09-12T09:29:55.204077+0000","last_active":"2024-09-12T09:30:31.870659+0000","last_peered":"2024-09-12T09:30:31.870659+0000","last_clean":"2024-09-12T09:30:31.870659+0000","last_became_active":"2024-09-12T09:29:55.135753+0000","last_became_peered":"2024-09-12T09:29:55.135753+0000","last_unstale":"2024-09-12T09:30:31.870659+0000","last_undegraded":"2024-09-12T09:30:31.870659+0000","last_fullsized":"2024-09-12T09:30:31.870659+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:03.925755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":21190526,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":4060,"num_read_kb":16362,"num_write":189,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":57888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":119,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.868088+0000","last_change":"2024-09-12T09:29:25.964568+0000","last_active":"2024-09-12T09:30:30.868088+0000","last_peered":"2024-09-12T09:30:30.868088+0000","last_clean":"2024-09-12T09:30:30.868088+0000","last_became_active":"2024-09-12T09:29:23.942494+0000","last_became_peered":"2024-09-12T09:29:23.942494+0000","last_unstale":"2024-09-12T09:30:30.868088+0000","last_undegraded":"2024-09-12T09:30:30.868088+0000","last_fullsized":"2024-09-12T09:30:30.868088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:12.205679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'109","reported_seq":2651,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.871272+0000","last_change":"2024-09-12T09:29:36.679214+0000","last_active":"2024-09-12T09:30:30.871272+0000","last_peered":"2024-09-12T09:30:30.871272+0000","last_clean":"2024-09-12T09:30:30.871272+0000","last_became_active":"2024-09-12T09:29:34.667231+0000","last_became_peered":"2024-09-12T09:29:34.667231+0000","last_unstale":"2024-09-12T09:30:30.871272+0000","last_undegraded":"2024-09-12T09:30:30.871272+0000","last_fullsized":"2024-09-12T09:30:30.871272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":28986728,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":2478,"num_read_kb":29377,"num_write":91,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.859565+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:30:30.859565+0000","last_peered":"2024-09-12T09:30:30.859565+0000","last_clean":"2024-09-12T09:30:30.859565+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:30:30.859565+0000","last_undegraded":"2024-09-12T09:30:30.859565+0000","last_fullsized":"2024-09-12T09:30:30.859565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"68'154","reported_seq":9561,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.860568+0000","last_change":"2024-09-12T09:29:55.163561+0000","last_active":"2024-09-12T09:30:30.860568+0000","last_peered":"2024-09-12T09:30:30.860568+0000","last_clean":"2024-09-12T09:30:30.860568+0000","last_became_active":"2024-09-12T09:29:55.134480+0000","last_became_peered":"2024-09-12T09:29:55.134480+0000","last_unstale":"2024-09-12T09:30:30.860568+0000","last_undegraded":"2024-09-12T09:30:30.860568+0000","last_fullsized":"2024-09-12T09:30:30.860568+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:06:19.510813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":36625257,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":10112,"num_read_kb":31810,"num_write":131,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2206098,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.933798+0000","last_change":"2024-09-12T09:30:30.876519+0000","last_active":"2024-09-12T09:30:31.933798+0000","last_peered":"2024-09-12T09:30:31.933798+0000","last_clean":"2024-09-12T09:30:31.933798+0000","last_became_active":"2024-09-12T09:30:30.875756+0000","last_became_peered":"2024-09-12T09:30:30.875756+0000","last_unstale":"2024-09-12T09:30:31.933798+0000","last_undegraded":"2024-09-12T09:30:31.933798+0000","last_fullsized":"2024-09-12T09:30:31.933798+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:54.962528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'88","reported_seq":4267,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.867716+0000","last_change":"2024-09-12T09:30:12.711941+0000","last_active":"2024-09-12T09:30:30.867716+0000","last_peered":"2024-09-12T09:30:30.867716+0000","last_clean":"2024-09-12T09:30:30.867716+0000","last_became_active":"2024-09-12T09:30:12.711307+0000","last_became_peered":"2024-09-12T09:30:12.711307+0000","last_unstale":"2024-09-12T09:30:30.867716+0000","last_undegraded":"2024-09-12T09:30:30.867716+0000","last_fullsized":"2024-09-12T09:30:30.867716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:35.859659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":29564344,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":4493,"num_read_kb":40553,"num_write":107,"num_write_kb":10031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":118,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.869298+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:30:30.869298+0000","last_peered":"2024-09-12T09:30:30.869298+0000","last_clean":"2024-09-12T09:30:30.869298+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:30:30.869298+0000","last_undegraded":"2024-09-12T09:30:30.869298+0000","last_fullsized":"2024-09-12T09:30:30.869298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.933531+0000","last_change":"2024-09-12T09:30:30.889337+0000","last_active":"2024-09-12T09:30:31.933531+0000","last_peered":"2024-09-12T09:30:31.933531+0000","last_clean":"2024-09-12T09:30:31.933531+0000","last_became_active":"2024-09-12T09:30:30.873745+0000","last_became_peered":"2024-09-12T09:30:30.873745+0000","last_unstale":"2024-09-12T09:30:31.933531+0000","last_undegraded":"2024-09-12T09:30:31.933531+0000","last_fullsized":"2024-09-12T09:30:31.933531+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:57.848261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"68'212","reported_seq":3373,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.864337+0000","last_change":"2024-09-12T09:29:36.673435+0000","last_active":"2024-09-12T09:30:30.864337+0000","last_peered":"2024-09-12T09:30:30.864337+0000","last_clean":"2024-09-12T09:30:30.864337+0000","last_became_active":"2024-09-12T09:29:34.665426+0000","last_became_peered":"2024-09-12T09:29:34.665426+0000","last_unstale":"2024-09-12T09:30:30.864337+0000","last_undegraded":"2024-09-12T09:30:30.864337+0000","last_fullsized":"2024-09-12T09:30:30.864337+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:56.125513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":37915960,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":4758,"num_read_kb":48386,"num_write":262,"num_write_kb":20369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'94","reported_seq":1603,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.869909+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:30:31.869909+0000","last_peered":"2024-09-12T09:30:31.869909+0000","last_clean":"2024-09-12T09:30:31.869909+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:30:31.869909+0000","last_undegraded":"2024-09-12T09:30:31.869909+0000","last_fullsized":"2024-09-12T09:30:31.869909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:04.664272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":12034925,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":1409,"num_read_kb":9307,"num_write":80,"num_write_kb":7160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"68'93","reported_seq":4160,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.869928+0000","last_change":"2024-09-12T09:30:01.232115+0000","last_active":"2024-09-12T09:30:31.869928+0000","last_peered":"2024-09-12T09:30:31.869928+0000","last_clean":"2024-09-12T09:30:31.869928+0000","last_became_active":"2024-09-12T09:30:01.179562+0000","last_became_peered":"2024-09-12T09:30:01.179562+0000","last_unstale":"2024-09-12T09:30:31.869928+0000","last_undegraded":"2024-09-12T09:30:31.869928+0000","last_fullsized":"2024-09-12T09:30:31.869928+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:52:23.416506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":18175791,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":4083,"num_read_kb":9333,"num_write":138,"num_write_kb":4065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2670405,"num_keys_recovered":16,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"63'219","reported_seq":5323,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.864484+0000","last_change":"2024-09-12T09:29:55.239142+0000","last_active":"2024-09-12T09:30:30.864484+0000","last_peered":"2024-09-12T09:30:30.864484+0000","last_clean":"2024-09-12T09:30:30.864484+0000","last_became_active":"2024-09-12T09:29:55.135093+0000","last_became_peered":"2024-09-12T09:29:55.135093+0000","last_unstale":"2024-09-12T09:30:30.864484+0000","last_undegraded":"2024-09-12T09:30:30.864484+0000","last_fullsized":"2024-09-12T09:30:30.864484+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:38.193128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":62848525,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5733,"num_read_kb":69702,"num_write":401,"num_write_kb":29356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1338871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'123","reported_seq":1418,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.868045+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:30:30.868045+0000","last_peered":"2024-09-12T09:30:30.868045+0000","last_clean":"2024-09-12T09:30:30.868045+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:30:30.868045+0000","last_undegraded":"2024-09-12T09:30:30.868045+0000","last_fullsized":"2024-09-12T09:30:30.868045+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:32:42.030701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":21836085,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":2191,"num_read_kb":22371,"num_write":153,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'127","reported_seq":6013,"reported_epoch":70,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:30.865343+0000","last_change":"2024-09-12T09:30:30.865343+0000","last_active":"2024-09-12T09:30:30.865343+0000","last_peered":"2024-09-12T09:30:28.892925+0000","last_clean":"2024-09-12T09:30:28.892925+0000","last_became_active":"2024-09-12T09:30:06.685061+0000","last_became_peered":"2024-09-12T09:30:06.685061+0000","last_unstale":"2024-09-12T09:30:30.865343+0000","last_undegraded":"2024-09-12T09:30:30.865343+0000","last_fullsized":"2024-09-12T09:30:30.865343+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":21692814,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":6345,"num_read_kb":22707,"num_write":195,"num_write_kb":1963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":557161,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"68'171","reported_seq":3789,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.874016+0000","last_change":"2024-09-12T09:30:27.834395+0000","last_active":"2024-09-12T09:30:30.874016+0000","last_peered":"2024-09-12T09:30:30.874016+0000","last_clean":"2024-09-12T09:30:30.874016+0000","last_became_active":"2024-09-12T09:30:27.833222+0000","last_became_peered":"2024-09-12T09:30:27.833222+0000","last_unstale":"2024-09-12T09:30:30.874016+0000","last_undegraded":"2024-09-12T09:30:30.874016+0000","last_fullsized":"2024-09-12T09:30:30.874016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:05:36.462698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":58194585,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":4242,"num_read_kb":87160,"num_write":216,"num_write_kb":42572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":55394924,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"68'153","reported_seq":5010,"reported_epoch":70,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:30.861172+0000","last_change":"2024-09-12T09:30:30.861172+0000","last_active":"2024-09-12T09:30:30.861172+0000","last_peered":"2024-09-12T09:30:28.989850+0000","last_clean":"2024-09-12T09:30:28.989850+0000","last_became_active":"2024-09-12T09:30:09.704611+0000","last_became_peered":"2024-09-12T09:30:09.704611+0000","last_unstale":"2024-09-12T09:30:30.861172+0000","last_undegraded":"2024-09-12T09:30:30.861172+0000","last_fullsized":"2024-09-12T09:30:30.861172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":53912818,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":5045,"num_read_kb":72237,"num_write":211,"num_write_kb":40553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":34086545,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"59'152","reported_seq":1070,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-12T09:30:30.859957+0000","last_change":"2024-09-12T09:30:03.657130+0000","last_active":"2024-09-12T09:30:30.859957+0000","last_peered":"2024-09-12T09:30:30.859957+0000","last_clean":"2024-09-12T09:30:30.859957+0000","last_became_active":"2024-09-12T09:30:03.655757+0000","last_became_peered":"2024-09-12T09:30:03.655757+0000","last_unstale":"2024-09-12T09:30:30.859957+0000","last_undegraded":"2024-09-12T09:30:30.859957+0000","last_fullsized":"2024-09-12T09:30:30.859957+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:26.191927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":26015975,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":1037,"num_read_kb":6021,"num_write":179,"num_write_kb":11663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":4010639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"71'90","reported_seq":94,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:32.050925+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:30:32.050925+0000","last_peered":"2024-09-12T09:30:32.050925+0000","last_clean":"2024-09-12T09:30:32.050925+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:30:32.050925+0000","last_undegraded":"2024-09-12T09:30:32.050925+0000","last_fullsized":"2024-09-12T09:30:32.050925+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:03.522697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":29419046,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":1646,"num_read_kb":8871,"num_write":108,"num_write_kb":703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'199","reported_seq":10337,"reported_epoch":69,"state":"remapped+peering","last_fresh":"2024-09-12T09:30:29.860999+0000","last_change":"2024-09-12T09:30:30.861825+0000","last_active":"2024-09-12T09:30:29.860999+0000","last_peered":"2024-09-12T09:30:28.484461+0000","last_clean":"2024-09-12T09:30:28.484461+0000","last_became_active":"2024-09-12T09:30:09.703273+0000","last_became_peered":"2024-09-12T09:30:09.703273+0000","last_unstale":"2024-09-12T09:30:29.860999+0000","last_undegraded":"2024-09-12T09:30:29.860999+0000","last_fullsized":"2024-09-12T09:30:29.860999+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":70366143,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":10814,"num_read_kb":92884,"num_write":302,"num_write_kb":53489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":43473478,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'63","reported_seq":55,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-12T09:30:31.933795+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:30:31.933795+0000","last_peered":"2024-09-12T09:30:31.933795+0000","last_clean":"2024-09-12T09:30:31.933795+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:30:31.933795+0000","last_undegraded":"2024-09-12T09:30:31.933795+0000","last_fullsized":"2024-09-12T09:30:31.933795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:23:32.098514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":33150263,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":401,"num_read_kb":3394,"num_write":158,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1030110362,"num_objects":905,"num_object_clones":189,"num_object_copies":1810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":398,"num_read":137962,"num_read_kb":1080862,"num_write":6201,"num_write_kb":620728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":189426082,"num_keys_recovered":85,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58155008,"data_stored":861079855,"data_compressed":5438411,"data_compressed_allocated":52355072,"data_compressed_original":856952986,"omap_allocated":0,"internal_metadata":0},"log_size":4628,"ondisk_log_size":4628,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":26616,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1880064,"data_stored":5169719,"data_compressed":180311,"data_compressed_allocated":901120,"data_compressed_original":4714496,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47192,"kb_used_data":11836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324648,"statfs":{"total":96636764160,"available":96588439552,"internally_reserved":0,"allocated":12120064,"data_stored":167528747,"data_compressed":1077729,"data_compressed_allocated":10166272,"data_compressed_original":166345113,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542163,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46928,"kb_used_data":11576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324912,"statfs":{"total":96636764160,"available":96588709888,"internally_reserved":0,"allocated":11853824,"data_stored":175847468,"data_compressed":711492,"data_compressed_allocated":9957376,"data_compressed_original":174648135,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51816,"kb_used_data":12308,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320024,"statfs":{"total":96636764160,"available":96583704576,"internally_reserved":0,"allocated":12603392,"data_stored":181041846,"data_compressed":1142020,"data_compressed_allocated":10891264,"data_compressed_original":180103864,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38568,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333272,"statfs":{"total":96636764160,"available":96597270528,"internally_reserved":0,"allocated":3297280,"data_stored":20244851,"data_compressed":475705,"data_compressed_allocated":2125824,"data_compressed_original":19671126,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39520,"kb_used_data":4172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332320,"statfs":{"total":96636764160,"available":96596295680,"internally_reserved":0,"allocated":4272128,"data_stored":36454607,"data_compressed":475051,"data_compressed_allocated":2973696,"data_compressed_original":35808157,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56388,"kb_used_data":16876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315452,"statfs":{"total":96636764160,"available":96579022848,"internally_reserved":0,"allocated":17281024,"data_stored":269683520,"data_compressed":1306856,"data_compressed_allocated":14905344,"data_compressed_original":268055823,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16052224,"data_stored":268120137,"data_compressed":1196401,"data_compressed_allocated":14602240,"data_compressed_original":266917135,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3428352,"data_stored":35504418,"data_compressed":468397,"data_compressed_allocated":2936832,"data_compressed_original":35218333,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2498560,"data_stored":19885049,"data_compressed":475705,"data_compressed_allocated":2125824,"data_compressed_original":19671126,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11804672,"data_stored":180682044,"data_compressed":1142020,"data_compressed_allocated":10891264,"data_compressed_original":180103864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11296768,"data_stored":167156652,"data_compressed":1077729,"data_compressed_allocated":10166272,"data_compressed_original":166345113,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11051008,"data_stored":175487647,"data_compressed":711492,"data_compressed_allocated":9957376,"data_compressed_original":174648135,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":4709750,"data_compressed":109990,"data_compressed_allocated":405504,"data_compressed_original":4681728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1556480,"data_stored":9534158,"data_compressed":256677,"data_compressed_allocated":1269760,"data_compressed_original":9367552,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:30:35.701 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [6, 7] 2024-09-12T09:30:35.701 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '6', '7'] 2024-09-12T09:30:35.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 6 7 2024-09-12T09:30:35.730 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:35.728+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:35.768 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.768 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:35.872 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:35.868+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:30:35.931 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:35.928+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:35.957 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:35.957 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:35.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:36.032 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:36.032+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:36.087 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:36.087 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:36.089 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone19 from 0xe1461e9 (overlap 0x61d623) to 0x0 2024-09-12T09:30:36.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:36.152 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:36.148+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:36.233 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:36.232+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:36.245 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:36.245 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:36.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:36.335 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:36.332+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:36.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:36.395 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:36.395 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:36.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:36.448 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-12T09:30:36.452 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:36.452+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:36.536 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:36.532+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:36.543 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:36.560 INFO:teuthology.orchestra.run.smithi007.stdout:1831 write 0xe493cf7 thru 0xe49bb52 (0x7e5c bytes) 2024-09-12T09:30:36.562 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:36.574 INFO:teuthology.orchestra.run.smithi007.stdout:1832 punch from 0xd4f1fd6 to 0xd4f3092, (0x10bc bytes) 2024-09-12T09:30:36.574 INFO:teuthology.orchestra.run.smithi007.stdout:1833 punch from 0x77265f2 to 0x7730f60, (0xa96e bytes) 2024-09-12T09:30:36.574 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:36.574 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:36.574 INFO:teuthology.orchestra.run.smithi007.stdout:1839 write 0xbd044c1 thru 0xbd0ce16 (0x8956 bytes) 2024-09-12T09:30:36.581 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:36.585 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:36.585 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:36.599 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:36.600 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:36.600 INFO:teuthology.orchestra.run.smithi007.stdout:1841 trunc from 0xe49bb53 to 0x63894c7 2024-09-12T09:30:36.601 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:36.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:1842 read 0x3708a80 thru 0x370e890 (0x5e11 bytes) 2024-09-12T09:30:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:1843 read 0x1599840 thru 0x15a4b0f (0xb2d0 bytes) 2024-09-12T09:30:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:1844 write 0x1f739ae thru 0x1f7de52 (0xa4a5 bytes) 2024-09-12T09:30:36.636 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:36.632+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:36.643 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:36.653 INFO:teuthology.orchestra.run.smithi007.stdout:1847 writesame 0xd906f52 thru 0xd90c55a data_size 0x5(0x5609 bytes) 2024-09-12T09:30:36.662 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:1848 trunc from 0xd90c55b to 0x50d669d 2024-09-12T09:30:36.686 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:36.686 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:1849 writesame 0x7f3bd1b thru 0x7f3db78 data_size 0x1a(0x1e5e bytes) 2024-09-12T09:30:36.696 INFO:teuthology.orchestra.run.smithi007.stdout:1850 clone 21 order 19 su 131072 sc 14 2024-09-12T09:30:36.705 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:36.705 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:36.735 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.445175108999997, 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:30:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.985678880000002, 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:30:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:30:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.612261928999999, 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:30:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:30:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.612214682000001, 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:30:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.611921605999999, 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:30:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:30:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.611359673999999, 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:30:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.611308310999998, 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:30:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:30:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.610735430999998, 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.610583131999999, 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:30:36.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:30:36.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.610226042000001, 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:30:36.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.609601072, 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:30:36.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:36.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.609549735999998, 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:36.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:30:36.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:30:36.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.608754946000001, 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:30:36.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.608680056000001, 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:30:36.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:30:36.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.608627735999999, 2024-09-12T09:30:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:30:36.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.608579138, 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:30:36.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:36.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.608324672999998, 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:30:36.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.604870383000002, 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:30:36.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:36.818 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.547301836000001, 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:36.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:30:36.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:30:36.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:30:36.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0760305990000001, 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:30:36.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:30:36.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:36.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:36.831 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:36.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:36.833 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:36.832+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.2 pg_upmap mapping to [6,7] 2024-09-12T09:30:36.932 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:36.928+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:37.016 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.017 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:37.017 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:37.017 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:37.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:37.033 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:37.028+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:37.135 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:37.128+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:37.135 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone20 intact 2024-09-12T09:30:37.155 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.155 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:37.155 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:37.155 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:37.155 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:37.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:37.235 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:37.232+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:37.290 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.290 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.878236270999999, 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:37.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.878141008999997, 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:37.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.874456385000002, 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:37.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.872651294999997, 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:37.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.872104620000002, 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:37.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:37.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.871336177000003, 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:37.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.870925808999999, 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:37.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.870786633000002, 2024-09-12T09:30:37.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:37.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.863007822, 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:37.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:37.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.860643807999999, 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:37.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.849891915000001, 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:37.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.849695523999998, 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:37.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.848201103999997, 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:37.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:37.354 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.147958243000001, 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:37.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.147935825000001, 2024-09-12T09:30:37.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:37.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.147912684000001, 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:37.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.147887725, 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:37.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:37.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:37.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.104536210999999, 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:30:37.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:30:37.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:30:37.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.488058169, 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:30:37.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.487958879000001, 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:30:37.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.388 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:37.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:37.390 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:37.388+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:37.488 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-12T09:30:37.490 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:37.488+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:37.491 INFO:teuthology.orchestra.run.smithi007.stdout:1851 writesame 0x426855e thru 0x426f4b0 data_size 0x1(0x6f53 bytes) 2024-09-12T09:30:37.560 INFO:teuthology.orchestra.run.smithi007.stdout:1852 writesame 0x9e5df1e thru 0x9e69d3f data_size 0x1(0xbe22 bytes) 2024-09-12T09:30:37.560 INFO:teuthology.orchestra.run.smithi007.stdout:1854 write 0x435e57b thru 0x435ffd8 (0x1a5e bytes) 2024-09-12T09:30:37.568 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:37.569 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:37.569 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:37.570 INFO:teuthology.orchestra.run.smithi007.stdout:1855 read 0x30f85ac thru 0x3106ef0 (0xe945 bytes) 2024-09-12T09:30:37.570 INFO:teuthology.orchestra.run.smithi007.stdout:1856 flatten 2024-09-12T09:30:37.589 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:37.590 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:37.588+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:37.691 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:37.688+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:37.787 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.787 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:37.788 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:37.788 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:37.788 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:37.792 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:37.788+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:37.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:37.892 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:37.892+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:37.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:30:37.963 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:37.963 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324642978999997, 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324593370999999, 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:37.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324559628999999, 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:37.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324512591000001, 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.978 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:37.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:37.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324454750999998, 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:37.982 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:37.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324395664000001, 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:37.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:37.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324361476999997, 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:37.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:37.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:37.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324330140999997, 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:37.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:37.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:37.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:38.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:38.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324300100000002, 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:38.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324268855, 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:38.006 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:38.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:38.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:38.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324227086, 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:38.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324195619999998, 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324148080000001, 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:38.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:38.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324115042999999, 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:38.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324070022999997, 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:38.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324037365000002, 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.324005550000003, 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:38.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:38.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.323947637000003, 2024-09-12T09:30:38.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:38.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:38.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.323905654999997, 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:38.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.323869672999997, 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:38.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:38.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.057 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:38.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:38.059 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:38.052+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:38.149 INFO:teuthology.orchestra.run.smithi007.stdout:1859 write 0x9620f84 thru 0x9626c41 (0x5cbe bytes) 2024-09-12T09:30:38.157 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.159 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:38.152+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:38.173 INFO:teuthology.orchestra.run.smithi007.stdout:1861 trunc from 0x9e69d40 to 0x2e2adc2 2024-09-12T09:30:38.228 INFO:teuthology.orchestra.run.smithi007.stdout:1865 write 0xd837fe6 thru 0xd83c356 (0x4371 bytes) 2024-09-12T09:30:38.231 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:38.231 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:38.231 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:38.231 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:38.245 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:38.253 INFO:teuthology.orchestra.run.smithi007.stdout:1866 trunc from 0xd83c357 to 0x4f61961 2024-09-12T09:30:38.260 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:38.256+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:38.274 INFO:teuthology.orchestra.run.smithi007.stdout:1867 write 0x3ff1025 thru 0x3ff1385 (0x361 bytes) 2024-09-12T09:30:38.280 INFO:teuthology.orchestra.run.smithi007.stdout:1868 punch from 0x2d540cb to 0x2d5b5f1, (0x7526 bytes) 2024-09-12T09:30:38.297 INFO:teuthology.orchestra.run.smithi007.stdout:1869 write 0x753cf46 thru 0x7548543 (0xb5fe bytes) 2024-09-12T09:30:38.303 INFO:teuthology.orchestra.run.smithi007.stdout:1874 read 0x2b5629b thru 0x2b612ff (0xb065 bytes) 2024-09-12T09:30:38.303 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.319 INFO:teuthology.orchestra.run.smithi007.stdout:1876 write 0x9896c9a thru 0x989be23 (0x518a bytes) 2024-09-12T09:30:38.324 INFO:teuthology.orchestra.run.smithi007.stdout:1877 writesame 0x79264bb thru 0x793166f data_size 0x43(0xb1b5 bytes) 2024-09-12T09:30:38.329 INFO:teuthology.orchestra.run.smithi007.stdout:1878 read 0x470979b thru 0x47120a4 (0x890a bytes) 2024-09-12T09:30:38.329 INFO:teuthology.orchestra.run.smithi007.stdout:1880 punch from 0x44d7c4d to 0x44e1985, (0x9d38 bytes) 2024-09-12T09:30:38.329 INFO:teuthology.orchestra.run.smithi007.stdout:1881 writesame 0x8b3cbcc thru 0x8b3ff52 data_size 0x1(0x3387 bytes) 2024-09-12T09:30:38.342 INFO:teuthology.orchestra.run.smithi007.stdout:1882 read 0x8085fce thru 0x808ca69 (0x6a9c bytes) 2024-09-12T09:30:38.342 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.342 INFO:teuthology.orchestra.run.smithi007.stdout:1887 read 0x54ea27 thru 0x551cd9 (0x32b3 bytes) 2024-09-12T09:30:38.352 INFO:teuthology.orchestra.run.smithi007.stdout:1888 trunc from 0x989be24 to 0x407052f 2024-09-12T09:30:38.361 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:38.360+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:38.371 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:38.386 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:38.388 INFO:teuthology.orchestra.run.smithi007.stdout:1891 write 0x66ddf8d thru 0x66dea11 (0xa85 bytes) 2024-09-12T09:30:38.411 INFO:teuthology.orchestra.run.smithi007.stdout:1892 writesame 0x87281e6 thru 0x872e2d7 data_size 0x1(0x60f2 bytes) 2024-09-12T09:30:38.411 INFO:teuthology.orchestra.run.smithi007.stdout:1893 punch from 0x3a87431 to 0x3a907e9, (0x93b8 bytes) 2024-09-12T09:30:38.411 INFO:teuthology.orchestra.run.smithi007.stdout:1895 writesame 0x247ff75 thru 0x248988b data_size 0x1(0x9917 bytes) 2024-09-12T09:30:38.449 INFO:teuthology.orchestra.run.smithi007.stdout:1896 punch from 0x2ba952e to 0x2bab145, (0x1c17 bytes) 2024-09-12T09:30:38.449 INFO:teuthology.orchestra.run.smithi007.stdout:1898 read 0x414d325 thru 0x415b110 (0xddec bytes) 2024-09-12T09:30:38.449 INFO:teuthology.orchestra.run.smithi007.stdout:1900 write 0x43cf847 thru 0x43d80de (0x8898 bytes) 2024-09-12T09:30:38.457 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.457 INFO:teuthology.orchestra.run.smithi007.stdout:1902 read 0x366e2b6 thru 0x367ce1a (0xeb65 bytes) 2024-09-12T09:30:38.458 INFO:teuthology.orchestra.run.smithi007.stdout:1903 write 0x5e535aa thru 0x5e58759 (0x51b0 bytes) 2024-09-12T09:30:38.472 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:38.472+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:38.478 INFO:teuthology.orchestra.run.smithi007.stdout:1904 trunc from 0x872e2d8 to 0xd3cf2a8 2024-09-12T09:30:38.491 INFO:teuthology.orchestra.run.smithi007.stdout:1905 read 0x6dbaa5e thru 0x6dbdf15 (0x34b8 bytes) 2024-09-12T09:30:38.491 INFO:teuthology.orchestra.run.smithi007.stdout:1907 read 0xb9110b8 thru 0xb91162b (0x574 bytes) 2024-09-12T09:30:38.491 INFO:teuthology.orchestra.run.smithi007.stdout:1909 trunc from 0xd3cf2a8 to 0xd336be9 2024-09-12T09:30:38.503 INFO:teuthology.orchestra.run.smithi007.stdout:1910 punch from 0x8451133 to 0x8451bc5, (0xa92 bytes) 2024-09-12T09:30:38.503 INFO:teuthology.orchestra.run.smithi007.stdout:1911 write 0xc1ab2d1 thru 0xc1ba22c (0xef5c bytes) 2024-09-12T09:30:38.510 INFO:teuthology.orchestra.run.smithi007.stdout:1912 writesame 0x34b9b74 thru 0x34ba096 data_size 0x1(0x523 bytes) 2024-09-12T09:30:38.535 INFO:teuthology.orchestra.run.smithi007.stdout:1913 writesame 0xe98b7da thru 0xe996641 data_size 0x4(0xae68 bytes) 2024-09-12T09:30:38.535 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.535 INFO:teuthology.orchestra.run.smithi007.stdout:1915 writesame 0x9b1180c thru 0x9b13efb data_size 0x38(0x26f0 bytes) 2024-09-12T09:30:38.541 INFO:teuthology.orchestra.run.smithi007.stdout:1916 read 0x955eb2d thru 0x956313b (0x460f bytes) 2024-09-12T09:30:38.541 INFO:teuthology.orchestra.run.smithi007.stdout:1917 writesame 0xb331fe3 thru 0xb33376f data_size 0x1(0x178d bytes) 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162532661999997, 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:30:38.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162473251999998, 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.566 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:38.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:38.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162416665999999, 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:38.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:38.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:38.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162348352999999, 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:38.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162290167000002, 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:38.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:38.584 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162229633999999, 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:38.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162169634000001, 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:38.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:38.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:38.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162113284999997, 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:38.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.162053974999999, 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.603 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161998341999997, 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:38.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161941959000004, 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:38.608 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:38.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161885388999998, 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:38.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161828972999999, 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.618 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:38.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161773480000001, 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:38.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:38.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161685466999998, 2024-09-12T09:30:38.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.161621023999999, 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:38.632 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:38.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:38.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.315173238, 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:38.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:38.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:38.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:38.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.131757053000001, 2024-09-12T09:30:38.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:38.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.01569095, 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:38.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:38.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.902661092000002, 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:38.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:38.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:38.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:38.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:38.662 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:1918 trunc from 0xe996642 to 0xb7ad31a 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:1919 write 0xe235b09 thru 0xe239cb7 (0x41af bytes) 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:1921 read 0xa2f825e thru 0xa2fcd2f (0x4ad2 bytes) 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:1922 trunc from 0xe239cb8 to 0x5618bed 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:1925 read 0x2b5d1c5 thru 0x2b5d682 (0x4be bytes) 2024-09-12T09:30:38.664 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:38.660+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:38.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:38.689 INFO:teuthology.orchestra.run.smithi007.stdout:1926 write 0xb4581cd thru 0xb45e77c (0x65b0 bytes) 2024-09-12T09:30:38.696 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.696 INFO:teuthology.orchestra.run.smithi007.stdout:1928 punch from 0x60f6ff5 to 0x60f73de, (0x3e9 bytes) 2024-09-12T09:30:38.696 INFO:teuthology.orchestra.run.smithi007.stdout:1930 punch from 0x5f5a54d to 0x5f5f10b, (0x4bbe bytes) 2024-09-12T09:30:38.698 INFO:teuthology.orchestra.run.smithi007.stdout:1931 trunc from 0xb45e77d to 0x9b4c806 2024-09-12T09:30:38.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:38.724 INFO:teuthology.orchestra.run.smithi007.stdout:1932 trunc from 0x9b4c806 to 0xc09e89b 2024-09-12T09:30:38.734 INFO:teuthology.orchestra.run.smithi007.stdout:1933 writesame 0x751b3c0 thru 0x7528b51 data_size 0x29(0xd792 bytes) 2024-09-12T09:30:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:1934 punch from 0xab95fd to 0xac37b1, (0xa1b4 bytes) 2024-09-12T09:30:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:1935 write 0x1148a8a thru 0x11492a4 (0x81b bytes) 2024-09-12T09:30:38.750 INFO:teuthology.orchestra.run.smithi007.stdout:1938 trunc from 0xc09e89b to 0x9ff5ac8 2024-09-12T09:30:38.759 INFO:teuthology.orchestra.run.smithi007.stdout:1939 read 0x4569110 thru 0x4577cb6 (0xeba7 bytes) 2024-09-12T09:30:38.759 INFO:teuthology.orchestra.run.smithi007.stdout:1940 writesame 0x6ffb38e thru 0x70093ff data_size 0x1(0xe072 bytes) 2024-09-12T09:30:38.764 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:38.764+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:38.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:38.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:38.807 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:38.807 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:38.813 INFO:teuthology.orchestra.run.smithi007.stdout:1941 read 0x87dfe05 thru 0x87e92d4 (0x94d0 bytes) 2024-09-12T09:30:38.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:38.829 INFO:teuthology.orchestra.run.smithi007.stdout:1942 writesame 0xcbda32d thru 0xcbe5c1e data_size 0x27(0xb8f2 bytes) 2024-09-12T09:30:38.829 INFO:teuthology.orchestra.run.smithi007.stdout:1944 punch from 0x45a8565 to 0x45b7f69, (0xfa04 bytes) 2024-09-12T09:30:38.841 INFO:teuthology.orchestra.run.smithi007.stdout:1945 trunc from 0xcbe5c1f to 0x55fe352 2024-09-12T09:30:38.858 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.858 INFO:teuthology.orchestra.run.smithi007.stdout:1948 read 0x41a2935 thru 0x41ababd (0x9189 bytes) 2024-09-12T09:30:38.865 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:38.860+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:38.878 INFO:teuthology.orchestra.run.smithi007.stdout:1949 write 0xc0e694c thru 0xc0e953c (0x2bf1 bytes) 2024-09-12T09:30:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:1950 punch from 0x801a495 to 0x80274e5, (0xd050 bytes) 2024-09-12T09:30:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:1954 read 0x8d9a21e thru 0x8d9a577 (0x35a bytes) 2024-09-12T09:30:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:1956 write 0x3dac705 thru 0x3dac750 (0x4c bytes) 2024-09-12T09:30:38.887 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:38.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:38.891 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.891 INFO:teuthology.orchestra.run.smithi007.stdout:1958 write 0x56de7f0 thru 0x56ea07a (0xb88b bytes) 2024-09-12T09:30:38.899 INFO:teuthology.orchestra.run.smithi007.stdout:1959 write 0x36dab1c thru 0x36e4281 (0x9766 bytes) 2024-09-12T09:30:38.907 INFO:teuthology.orchestra.run.smithi007.stdout:1961 read 0xbd643e9 thru 0xbd6f15c (0xad74 bytes) 2024-09-12T09:30:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:30:38.913 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:38.916 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:38.916 INFO:teuthology.orchestra.run.smithi007.stdout:1963 write 0x8a4310e thru 0x8a4f2d8 (0xc1cb bytes) 2024-09-12T09:30:38.924 INFO:teuthology.orchestra.run.smithi007.stdout:1964 read 0x4ecaa1d thru 0x4ed8081 (0xd665 bytes) 2024-09-12T09:30:38.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:38.931 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:38.935 INFO:teuthology.orchestra.run.smithi007.stdout:1965 write 0xcad36c3 thru 0xcad5ef6 (0x2834 bytes) 2024-09-12T09:30:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:1966 trunc from 0xcad5ef7 to 0x8cbcd44 2024-09-12T09:30:38.951 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:38.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:38.964 INFO:teuthology.orchestra.run.smithi007.stdout:1967 punch from 0x3fbca40 to 0x3fc9d63, (0xd323 bytes) 2024-09-12T09:30:38.966 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:38.964+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:38.967 INFO:teuthology.orchestra.run.smithi007.stdout:1968 trunc from 0x8cbcd44 to 0xb4c2ca3 2024-09-12T09:30:38.975 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:38.976 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:38.976 INFO:teuthology.orchestra.run.smithi007.stdout:1970 read 0x59aad34 thru 0x59af9f9 (0x4cc6 bytes) 2024-09-12T09:30:38.976 INFO:teuthology.orchestra.run.smithi007.stdout:1971 write 0x6f3c9de thru 0x6f45eab (0x94ce bytes) 2024-09-12T09:30:38.977 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:38.977 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:38.984 INFO:teuthology.orchestra.run.smithi007.stdout:1972 read 0x7e7a860 thru 0x7e86fe4 (0xc785 bytes) 2024-09-12T09:30:38.984 INFO:teuthology.orchestra.run.smithi007.stdout:1973 write 0x5e43dd7 thru 0x5e4f8fd (0xbb27 bytes) 2024-09-12T09:30:38.992 INFO:teuthology.orchestra.run.smithi007.stdout:1974 punch from 0x7b0ae99 to 0x7b0fcda, (0x4e41 bytes) 2024-09-12T09:30:38.992 INFO:teuthology.orchestra.run.smithi007.stdout:1975 read 0x8c4ce6d thru 0x8c5103a (0x41ce bytes) 2024-09-12T09:30:38.992 INFO:teuthology.orchestra.run.smithi007.stdout:1978 punch from 0x30c2c1 to 0x316606, (0xa345 bytes) 2024-09-12T09:30:38.992 INFO:teuthology.orchestra.run.smithi007.stdout:1980 write 0x164155 thru 0x1694f7 (0x53a3 bytes) 2024-09-12T09:30:38.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:38.999 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:39.003 INFO:teuthology.orchestra.run.smithi007.stdout:1981 write 0xb8cc254 thru 0xb8d03e3 (0x4190 bytes) 2024-09-12T09:30:39.008 INFO:teuthology.orchestra.run.smithi007.stdout:1982 punch from 0x6a8e3bb to 0x6a953ca, (0x700f bytes) 2024-09-12T09:30:39.008 INFO:teuthology.orchestra.run.smithi007.stdout:1984 write 0xa2d3cee thru 0xa2d5da0 (0x20b3 bytes) 2024-09-12T09:30:39.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:39.028 INFO:teuthology.orchestra.run.smithi007.stdout:1985 trunc from 0xb8d03e4 to 0x2eef72c 2024-09-12T09:30:39.046 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:39.046 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:39.048 INFO:teuthology.orchestra.run.smithi007.stdout:1987 read 0x266e9ee thru 0x266fe5d (0x1470 bytes) 2024-09-12T09:30:39.058 INFO:teuthology.orchestra.run.smithi007.stdout:1988 writesame 0x490b45a thru 0x490dcee data_size 0x1(0x2895 bytes) 2024-09-12T09:30:39.067 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:39.060+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:39.070 INFO:teuthology.orchestra.run.smithi007.stdout:1989 trunc from 0x490dcef to 0xd0c8ae3 2024-09-12T09:30:39.084 INFO:teuthology.orchestra.run.smithi007.stdout:1991 write 0xd0ce6fc thru 0xd0d5102 (0x6a07 bytes) 2024-09-12T09:30:39.097 INFO:teuthology.orchestra.run.smithi007.stdout:1992 write 0xda9eff1 thru 0xdaa11fe (0x220e bytes) 2024-09-12T09:30:39.104 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:39.104 INFO:teuthology.orchestra.run.smithi007.stdout:1995 write 0xd38235 thru 0xd3c96f (0x473b bytes) 2024-09-12T09:30:39.111 INFO:teuthology.orchestra.run.smithi007.stdout:1996 read 0xd1c177 thru 0xd25dff (0x9c89 bytes) 2024-09-12T09:30:39.112 INFO:teuthology.orchestra.run.smithi007.stdout:1997 punch from 0xaee826a to 0xaeed781, (0x5517 bytes) 2024-09-12T09:30:39.112 INFO:teuthology.orchestra.run.smithi007.stdout:1998 write 0x382a5c5 thru 0x382b2fd (0xd39 bytes) 2024-09-12T09:30:39.117 INFO:teuthology.orchestra.run.smithi007.stdout:1999 write 0x4cebb4b thru 0x4cf2684 (0x6b3a bytes) 2024-09-12T09:30:39.124 INFO:teuthology.orchestra.run.smithi007.stdout:2000 write 0x6a1801d thru 0x6a1a819 (0x27fd bytes) 2024-09-12T09:30:39.129 INFO:teuthology.orchestra.run.smithi007.stdout:2001 read 0xa66d58a thru 0xa673559 (0x5fd0 bytes) 2024-09-12T09:30:39.129 INFO:teuthology.orchestra.run.smithi007.stdout:2006 write 0x5d00869 thru 0x5d07442 (0x6bda bytes) 2024-09-12T09:30:39.131 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:39.131 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:39.136 INFO:teuthology.orchestra.run.smithi007.stdout:2007 punch from 0x8b4c537 to 0x8b4cafb, (0x5c4 bytes) 2024-09-12T09:30:39.136 INFO:teuthology.orchestra.run.smithi007.stdout:2009 read 0xd341ac0 thru 0xd34fbc4 (0xe105 bytes) 2024-09-12T09:30:39.136 INFO:teuthology.orchestra.run.smithi007.stdout:2010 clone 22 order 24 su 262144 sc 13 2024-09-12T09:30:39.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:39.168 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:39.168+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:39.269 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:39.267+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:39.311 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:39.311 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:39.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:39.372 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:39.372+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:39.456 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:39.457 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:39.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:39.471 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:39.467+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:39.572 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:39.568+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:39.639 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:39.639 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:39.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:39.673 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:39.671+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:39.774 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:39.768+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:39.829 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:39.829 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:39.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:39.884 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:39.884+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:39.976 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:39.972+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:40.080 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:40.080+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:40.178 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:40.175+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:40.220 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.221 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:40.280 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:40.276+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:40.368 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-12T09:30:40.369 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.370 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.380 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:40.379+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:40.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:40.481 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:40.479+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:40.584 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:40.579+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:40.609 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.609 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:40.683 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:40.679+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:40.769 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.769 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.784 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:40.780+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:40.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:40.885 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:40.884+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:40.910 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:40.910 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:40.920 INFO:teuthology.orchestra.run.smithi007.stdout:2011 read 0x6945bff thru 0x694940a (0x380c bytes) 2024-09-12T09:30:40.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:40.925 INFO:teuthology.orchestra.run.smithi007.stdout:2012 writesame 0xb7e7e13 thru 0xb7f79c7 data_size 0x2f(0xfbb5 bytes) 2024-09-12T09:30:40.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:30:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:2013 read 0xb39ed50 thru 0xb3a0c8f (0x1f40 bytes) 2024-09-12T09:30:40.937 INFO:teuthology.orchestra.run.smithi007.stdout:2014 write 0x7ab2d9a thru 0x7ab37af (0xa16 bytes) 2024-09-12T09:30:40.943 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:40.943 INFO:teuthology.orchestra.run.smithi007.stdout:2017 punch from 0x96f87bb to 0x96fbd99, (0x35de bytes) 2024-09-12T09:30:40.943 INFO:teuthology.orchestra.run.smithi007.stdout:2018 read 0x3d58a9a thru 0x3d5e72f (0x5c96 bytes) 2024-09-12T09:30:40.943 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:40.944 INFO:teuthology.orchestra.run.smithi007.stdout:2021 punch from 0x8932c55 to 0x8941405, (0xe7b0 bytes) 2024-09-12T09:30:40.944 INFO:teuthology.orchestra.run.smithi007.stdout:2024 writesame 0x579dd04 thru 0x57a8747 data_size 0x22(0xaa44 bytes) 2024-09-12T09:30:40.951 INFO:teuthology.orchestra.run.smithi007.stdout:2025 write 0x6088841 thru 0x608f09a (0x685a bytes) 2024-09-12T09:30:40.970 INFO:teuthology.orchestra.run.smithi007.stdout:2027 trunc from 0xdaa11ff to 0x3164e57 2024-09-12T09:30:40.986 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:40.984+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:41.001 INFO:teuthology.orchestra.run.smithi007.stdout:2030 trunc from 0x3164e57 to 0x3ce61d0 2024-09-12T09:30:41.008 INFO:teuthology.orchestra.run.smithi007.stdout:2031 read 0x205d60b thru 0x20672b2 (0x9ca8 bytes) 2024-09-12T09:30:41.008 INFO:teuthology.orchestra.run.smithi007.stdout:2033 read 0x24f90e7 thru 0x24fa459 (0x1373 bytes) 2024-09-12T09:30:41.009 INFO:teuthology.orchestra.run.smithi007.stdout:2034 writesame 0x24cb664 thru 0x24d6546 data_size 0x1(0xaee3 bytes) 2024-09-12T09:30:41.045 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:41.046 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:41.048 INFO:teuthology.orchestra.run.smithi007.stdout:2035 writesame 0x88f2cc7 thru 0x88f71b5 data_size 0x1(0x44ef bytes) 2024-09-12T09:30:41.050 INFO:teuthology.orchestra.run.smithi007.stdout:2036 write 0x88c42bd thru 0x88cb00f (0x6d53 bytes) 2024-09-12T09:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout:2037 read 0x54a541f thru 0x54ac0b2 (0x6c94 bytes) 2024-09-12T09:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout:2041 read 0x87c255c thru 0x87c7610 (0x50b5 bytes) 2024-09-12T09:30:41.054 INFO:teuthology.orchestra.run.smithi007.stdout:2042 read 0x82f2e9d thru 0x82fcdab (0x9f0f bytes) 2024-09-12T09:30:41.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:41.064 INFO:teuthology.orchestra.run.smithi007.stdout:2046 write 0xa05ba5b thru 0xa06316b (0x7711 bytes) 2024-09-12T09:30:41.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:41.071 INFO:teuthology.orchestra.run.smithi007.stdout:2047 trunc from 0xa06316c to 0xcd0d38c 2024-09-12T09:30:41.079 INFO:teuthology.orchestra.run.smithi007.stdout:2048 read 0x43cbfcc thru 0x43d4d09 (0x8d3e bytes) 2024-09-12T09:30:41.087 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:41.084+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:41.087 INFO:teuthology.orchestra.run.smithi007.stdout:2050 write 0xd803e0f thru 0xd8058cd (0x1abf bytes) 2024-09-12T09:30:41.092 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.092 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.092 INFO:teuthology.orchestra.run.smithi007.stdout:2054 read 0x4fbffeb thru 0x4fcb62c (0xb642 bytes) 2024-09-12T09:30:41.113 INFO:teuthology.orchestra.run.smithi007.stdout:2055 trunc from 0xd8058ce to 0xfeff56 2024-09-12T09:30:41.145 INFO:teuthology.orchestra.run.smithi007.stdout:2058 trunc from 0xfeff56 to 0xab8a6a4 2024-09-12T09:30:41.152 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.157 INFO:teuthology.orchestra.run.smithi007.stdout:2061 trunc from 0xab8a6a4 to 0xdb72ccf 2024-09-12T09:30:41.173 INFO:teuthology.orchestra.run.smithi007.stdout:2062 write 0xe4b5846 thru 0xe4b68eb (0x10a6 bytes) 2024-09-12T09:30:41.178 INFO:teuthology.orchestra.run.smithi007.stdout:2063 read 0xa1a6130 thru 0xa1b4ecb (0xed9c bytes) 2024-09-12T09:30:41.178 INFO:teuthology.orchestra.run.smithi007.stdout:2064 read 0x30afb07 thru 0x30bb394 (0xb88e bytes) 2024-09-12T09:30:41.179 INFO:teuthology.orchestra.run.smithi007.stdout:2068 writesame 0x6eb00d thru 0x6ee9e5 data_size 0x1(0x39d9 bytes) 2024-09-12T09:30:41.180 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:41.180 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:41.188 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:41.184+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:41.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:41.196 INFO:teuthology.orchestra.run.smithi007.stdout:2069 write 0x5ee791b thru 0x5ef4d0d (0xd3f3 bytes) 2024-09-12T09:30:41.200 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.200 INFO:teuthology.orchestra.run.smithi007.stdout:2071 write 0xb230ebe thru 0xb23451a (0x365d bytes) 2024-09-12T09:30:41.202 INFO:teuthology.orchestra.run.smithi007.stdout:2072 write 0xa5a316d thru 0xa5b2f8b (0xfe1f bytes) 2024-09-12T09:30:41.222 INFO:teuthology.orchestra.run.smithi007.stdout:2073 trunc from 0xe4b68ec to 0x244d042 2024-09-12T09:30:41.241 INFO:teuthology.orchestra.run.smithi007.stdout:2074 read 0x77aadb thru 0x783024 (0x854a bytes) 2024-09-12T09:30:41.258 INFO:teuthology.orchestra.run.smithi007.stdout:2076 trunc from 0x244d042 to 0xe9456e7 2024-09-12T09:30:41.259 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:41.259 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:41.264 INFO:teuthology.orchestra.run.smithi007.stdout:2077 write 0xd531064 thru 0xd53126c (0x209 bytes) 2024-09-12T09:30:41.280 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:41.285 INFO:teuthology.orchestra.run.smithi007.stdout:2078 trunc from 0xe9456e7 to 0x2ca398c 2024-09-12T09:30:41.288 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:41.287+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:41.302 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:41.303 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:41.315 INFO:teuthology.orchestra.run.smithi007.stdout:2080 writesame 0x671bc74 thru 0x67236b1 data_size 0x1(0x7a3e bytes) 2024-09-12T09:30:41.323 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:41.323 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:41.324 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:41.324 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:41.325 INFO:teuthology.orchestra.run.smithi007.stdout:2081 writesame 0x6a41769 thru 0x6a4f1df data_size 0x1(0xda77 bytes) 2024-09-12T09:30:41.328 INFO:teuthology.orchestra.run.smithi007.stdout:2086 read 0x51985d7 thru 0x519e511 (0x5f3b bytes) 2024-09-12T09:30:41.328 INFO:teuthology.orchestra.run.smithi007.stdout:2087 punch from 0x45c766e to 0x45ca3a4, (0x2d36 bytes) 2024-09-12T09:30:41.328 INFO:teuthology.orchestra.run.smithi007.stdout:2088 read 0x6d2d03 thru 0x6d5a6d (0x2d6b bytes) 2024-09-12T09:30:41.329 INFO:teuthology.orchestra.run.smithi007.stdout:2089 read 0x118a3e5 thru 0x118fe1d (0x5a39 bytes) 2024-09-12T09:30:41.330 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.333 INFO:teuthology.orchestra.run.smithi007.stdout:2092 trunc from 0x6a4f1e0 to 0x8d40c24 2024-09-12T09:30:41.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:41.340 INFO:teuthology.orchestra.run.smithi007.stdout:2093 punch from 0x2c1592b to 0x2c19115, (0x37ea bytes) 2024-09-12T09:30:41.340 INFO:teuthology.orchestra.run.smithi007.stdout:2094 read 0x2f7b815 thru 0x2f865bb (0xada7 bytes) 2024-09-12T09:30:41.343 INFO:teuthology.orchestra.run.smithi007.stdout:2095 trunc from 0x8d40c24 to 0x6782680 2024-09-12T09:30:41.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:41.345 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:41.352 INFO:teuthology.orchestra.run.smithi007.stdout:2097 writesame 0x43901d3 thru 0x4391ae8 data_size 0x1a(0x1916 bytes) 2024-09-12T09:30:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:2098 punch from 0x54faa32 to 0x54feebc, (0x448a bytes) 2024-09-12T09:30:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:2099 write 0x3e5a574 thru 0x3e6856a (0xdff7 bytes) 2024-09-12T09:30:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:2101 write 0x5c9aede thru 0x5ca83f0 (0xd513 bytes) 2024-09-12T09:30:41.358 INFO:teuthology.orchestra.run.smithi007.stdout:2104 trunc from 0x6782680 to 0x67dee12 2024-09-12T09:30:41.363 INFO:teuthology.orchestra.run.smithi007.stdout:2106 write 0x3ce150 thru 0x3dd2dd (0xf18e bytes) 2024-09-12T09:30:41.366 INFO:teuthology.orchestra.run.smithi007.stdout:2107 read 0x3779de2 thru 0x377c364 (0x2583 bytes) 2024-09-12T09:30:41.366 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:41.367 INFO:teuthology.orchestra.run.smithi007.stdout:2108 read 0x16ba2a3 thru 0x16c6436 (0xc194 bytes) 2024-09-12T09:30:41.367 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.367 INFO:teuthology.orchestra.run.smithi007.stdout:2111 punch from 0x59feefb to 0x5a0d93e, (0xea43 bytes) 2024-09-12T09:30:41.367 INFO:teuthology.orchestra.run.smithi007.stdout:2112 punch from 0x19a40ca to 0x19b0605, (0xc53b bytes) 2024-09-12T09:30:41.374 INFO:teuthology.orchestra.run.smithi007.stdout:2113 trunc from 0x67dee12 to 0x1abb2b6 2024-09-12T09:30:41.388 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:41.402 INFO:teuthology.orchestra.run.smithi007.stdout:2114 writesame 0xa868b35 thru 0xa878a58 data_size 0x6(0xff24 bytes) 2024-09-12T09:30:41.408 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:41.414 INFO:teuthology.orchestra.run.smithi007.stdout:2116 write 0xcdf8d2f thru 0xce07746 (0xea18 bytes) 2024-09-12T09:30:41.415 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.416 INFO:teuthology.orchestra.run.smithi007.stdout:2118 read 0xae85968 thru 0xae8e2f7 (0x8990 bytes) 2024-09-12T09:30:41.416 INFO:teuthology.orchestra.run.smithi007.stdout:2119 write 0x7e13350 thru 0x7e17b52 (0x4803 bytes) 2024-09-12T09:30:41.418 INFO:teuthology.orchestra.run.smithi007.stdout:2122 punch from 0x35a2d8d to 0x35a5c29, (0x2e9c bytes) 2024-09-12T09:30:41.418 INFO:teuthology.orchestra.run.smithi007.stdout:2123 writesame 0xc1c6c6 thru 0xc2c163 data_size 0x4a(0xfa9e bytes) 2024-09-12T09:30:41.420 INFO:teuthology.orchestra.run.smithi007.stdout:2125 writesame 0x543cfc7 thru 0x54446fe data_size 0x6d(0x7738 bytes) 2024-09-12T09:30:41.431 INFO:teuthology.orchestra.run.smithi007.stdout:2126 write 0xe19d94c thru 0xe1a0aa7 (0x315c bytes) 2024-09-12T09:30:41.432 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:41.431+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:41.433 INFO:teuthology.orchestra.run.smithi007.stdout:2127 read 0xccf528a thru 0xcd043f9 (0xf170 bytes) 2024-09-12T09:30:41.434 INFO:teuthology.orchestra.run.smithi007.stdout:2128 write 0x761fa86 thru 0x7629cf0 (0xa26b bytes) 2024-09-12T09:30:41.435 INFO:teuthology.orchestra.run.smithi007.stdout:2129 write 0x74fe92e thru 0x7502ac3 (0x4196 bytes) 2024-09-12T09:30:41.437 INFO:teuthology.orchestra.run.smithi007.stdout:2131 read 0x5de99b8 thru 0x5df3c09 (0xa252 bytes) 2024-09-12T09:30:41.438 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.438 INFO:teuthology.orchestra.run.smithi007.stdout:2133 write 0x1e90a85 thru 0x1e90b17 (0x93 bytes) 2024-09-12T09:30:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:2135 punch from 0xbfcfd48 to 0xbfde639, (0xe8f1 bytes) 2024-09-12T09:30:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:2136 writesame 0x9ce0b92 thru 0x9ce247d data_size 0x1d(0x18ec bytes) 2024-09-12T09:30:41.441 INFO:teuthology.orchestra.run.smithi007.stdout:2137 read 0xc58fd26 thru 0xc59f2b3 (0xf58e bytes) 2024-09-12T09:30:41.441 INFO:teuthology.orchestra.run.smithi007.stdout:2138 writesame 0x64486c9 thru 0x644f402 data_size 0x3e(0x6d3a bytes) 2024-09-12T09:30:41.443 INFO:teuthology.orchestra.run.smithi007.stdout:2140 write 0x8ffecd5 thru 0x8fffca7 (0xfd3 bytes) 2024-09-12T09:30:41.445 INFO:teuthology.orchestra.run.smithi007.stdout:2141 read 0xa0f401a thru 0xa101935 (0xd91c bytes) 2024-09-12T09:30:41.446 INFO:teuthology.orchestra.run.smithi007.stdout:2142 writesame 0x947656 thru 0x95037d data_size 0x4(0x8d28 bytes) 2024-09-12T09:30:41.452 INFO:teuthology.orchestra.run.smithi007.stdout:2143 read 0x8c40bba thru 0x8c46a52 (0x5e99 bytes) 2024-09-12T09:30:41.452 INFO:teuthology.orchestra.run.smithi007.stdout:2144 writesame 0x1249ec1 thru 0x1254c69 data_size 0x7(0xada9 bytes) 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:2145 punch from 0x5165a74 to 0x516faba, (0xa046 bytes) 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:2146 punch from 0x9372d73 to 0x937e05e, (0xb2eb bytes) 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:2148 punch from 0xe17688e to 0xe177471, (0xbe3 bytes) 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:2150 read 0x1f78c06 thru 0x1f84770 (0xbb6b bytes) 2024-09-12T09:30:41.458 INFO:teuthology.orchestra.run.smithi007.stdout:2151 writesame 0x3ffb7a0 thru 0x3ffd157 data_size 0x4(0x19b8 bytes) 2024-09-12T09:30:41.460 INFO:teuthology.orchestra.run.smithi007.stdout:2153 write 0x26177aa thru 0x2625330 (0xdb87 bytes) 2024-09-12T09:30:41.470 INFO:teuthology.orchestra.run.smithi007.stdout:2154 writesame 0xe698f22 thru 0xe69afda data_size 0x1(0x20b9 bytes) 2024-09-12T09:30:41.471 INFO:teuthology.orchestra.run.smithi007.stdout:2157 read 0x6cdc3b4 thru 0x6cde25e (0x1eab bytes) 2024-09-12T09:30:41.474 INFO:teuthology.orchestra.run.smithi007.stdout:2158 trunc from 0xe69afdb to 0xc1e9323 2024-09-12T09:30:41.490 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:41.488+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:41.491 INFO:teuthology.orchestra.run.smithi007.stdout:2161 write 0x13d5fbe thru 0x13e2e03 (0xce46 bytes) 2024-09-12T09:30:41.493 INFO:teuthology.orchestra.run.smithi007.stdout:2162 read 0x8dbc449 thru 0x8dca6f1 (0xe2a9 bytes) 2024-09-12T09:30:41.494 INFO:teuthology.orchestra.run.smithi007.stdout:2163 read 0x4299ce8 thru 0x42a118a (0x74a3 bytes) 2024-09-12T09:30:41.494 INFO:teuthology.orchestra.run.smithi007.stdout:2164 trunc from 0xc1e9323 to 0xbf87fc6 2024-09-12T09:30:41.504 INFO:teuthology.orchestra.run.smithi007.stdout:2167 writesame 0x3b10b96 thru 0x3b1cd73 data_size 0xe(0xc1de bytes) 2024-09-12T09:30:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:2171 read 0x95ad70c thru 0x95b181d (0x4112 bytes) 2024-09-12T09:30:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:2173 punch from 0xc5ce53 to 0xc6b3af, (0xe55c bytes) 2024-09-12T09:30:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:2174 punch from 0x37397bc to 0x3743c6f, (0xa4b3 bytes) 2024-09-12T09:30:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:2175 writesame 0xa465003 thru 0xa46b65b data_size 0x13(0x6659 bytes) 2024-09-12T09:30:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:2176 trunc from 0xbf87fc6 to 0xc4847c2 2024-09-12T09:30:41.513 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:41.513 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:41.513 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:41.513 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:41.513 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:41.517 INFO:teuthology.orchestra.run.smithi007.stdout:2177 writesame 0x77a4291 thru 0x77b1079 data_size 0x3(0xcde9 bytes) 2024-09-12T09:30:41.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:41.528 INFO:teuthology.orchestra.run.smithi007.stdout:2178 read 0xc085b2f thru 0xc088e36 (0x3308 bytes) 2024-09-12T09:30:41.529 INFO:teuthology.orchestra.run.smithi007.stdout:2179 write 0x580bf77 thru 0x581676f (0xa7f9 bytes) 2024-09-12T09:30:41.532 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.532 INFO:teuthology.orchestra.run.smithi007.stdout:2181 read 0x451749d thru 0x4520984 (0x94e8 bytes) 2024-09-12T09:30:41.533 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.533 INFO:teuthology.orchestra.run.smithi007.stdout:2183 read 0x1001640 thru 0x100a8d7 (0x9298 bytes) 2024-09-12T09:30:41.534 INFO:teuthology.orchestra.run.smithi007.stdout:2184 write 0x3d7d64f thru 0x3d87cf7 (0xa6a9 bytes) 2024-09-12T09:30:41.540 INFO:teuthology.orchestra.run.smithi007.stdout:2185 trunc from 0xc4847c2 to 0x9f0d7c4 2024-09-12T09:30:41.561 INFO:teuthology.orchestra.run.smithi007.stdout:2187 write 0xb2c36c7 thru 0xb2d1502 (0xde3c bytes) 2024-09-12T09:30:41.564 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.564 INFO:teuthology.orchestra.run.smithi007.stdout:2189 write 0x45328c2 thru 0x453e443 (0xbb82 bytes) 2024-09-12T09:30:41.565 INFO:teuthology.orchestra.run.smithi007.stdout:2190 read 0x7e8c260 thru 0x7e8d7d6 (0x1577 bytes) 2024-09-12T09:30:41.573 INFO:teuthology.orchestra.run.smithi007.stdout:2191 trunc from 0xb2d1503 to 0x77987cb 2024-09-12T09:30:41.587 INFO:teuthology.orchestra.run.smithi007.stdout:2192 read 0x37a738b thru 0x37b247f (0xb0f5 bytes) 2024-09-12T09:30:41.602 INFO:teuthology.orchestra.run.smithi007.stdout:2193 trunc from 0x77987cb to 0xe200786 2024-09-12T09:30:41.611 INFO:teuthology.orchestra.run.smithi007.stdout:2194 trunc from 0xe200786 to 0xe260282 2024-09-12T09:30:41.616 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:41.615+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:41.619 INFO:teuthology.orchestra.run.smithi007.stdout:2195 write 0x51d2e27 thru 0x51e0a65 (0xdc3f bytes) 2024-09-12T09:30:41.622 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:41.622 INFO:teuthology.orchestra.run.smithi007.stdout:2198 read 0x4a5f0cd thru 0x4a60d8c (0x1cc0 bytes) 2024-09-12T09:30:41.623 INFO:teuthology.orchestra.run.smithi007.stdout:2199 clone 23 order 20 su 524288 sc 11 2024-09-12T09:30:41.692 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:41.688+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.697 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.284552067999996, 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.698 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:30:41.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:41.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.284456806000001, 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:30:41.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:30:41.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.280772182, 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:30:41.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.278967092000002, 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:30:41.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:30:41.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.278420417, 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:30:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.277651974000001, 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:30:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.277241605999997, 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:30:41.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.277102429999999, 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.739 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:30:41.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.269323618999998, 2024-09-12T09:30:41.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.266959604999997, 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:30:41.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:30:41.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.256207711999998, 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:41.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:30:41.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.256011321000003, 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:30:41.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.254516901000002, 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:30:41.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.554274039999999, 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:41.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.554251621999999, 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:30:41.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:30:41.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.781 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.554228480999999, 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:30:41.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:30:41.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.554203522000002, 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:30:41.786 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:41.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.510852008000001, 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:30:41.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:30:41.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:30:41.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.894373966, 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:30:41.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:30:41.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.894274675999998, 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:41.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:30:41.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:30:41.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:41.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:41.808 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:41.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:41.810 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:41.807+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:41.911 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:41.908+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:41.924 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:41.924 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:41.924 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 2, 3, 4, 5, 0, 7] 2024-09-12T09:30:41.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-12T09:30:41.932 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:30:41.982 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:41.982 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:41.982 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:41.982 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:41.997 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:42.033 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone22 intact 2024-09-12T09:30:42.044 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:42.043+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:42.113 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:42.111+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:42.120 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:42.120 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:42.121 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:42.121 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:42.121 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:42.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:42.220 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:42.219+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:42.254 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-12T09:30:42.268 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:30:42.268+0000 7f3454e55640 -1 mon.a@0(leader).osd e79 definitely_dead 0 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665930412999998, 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:42.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:42.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665880805, 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:42.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665847063000001, 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:42.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665800025000003, 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:42.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665742184999999, 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:42.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:42.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:42.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665683098000002, 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:42.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:42.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665648910999998, 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:42.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:42.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665617574999999, 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:42.341 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:42.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665587533999997, 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:42.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665556289000001, 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:42.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:42.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:42.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665514520000002, 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:42.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:42.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665483053999999, 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:42.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.365 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665435514000002, 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:42.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665402477000001, 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:42.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665357456999999, 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:42.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:42.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665324798999997, 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:42.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:42.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665292983999997, 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:42.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:42.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:42.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665235070999998, 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:42.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:42.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665193088999999, 2024-09-12T09:30:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:42.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.665157106999999, 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:42.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.402 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:42.404 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:42.404 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:42.404+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:42.505 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:42.504+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:42.574 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:42.574 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:42.575 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:42.575 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:42.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:42.606 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:42.604+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:42.705 INFO:teuthology.orchestra.run.smithi007.stdout:2200 write 0x631017e thru 0x631312d (0x2fb0 bytes) 2024-09-12T09:30:42.707 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:42.704+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:42.713 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:42.713 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:42.713 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:42.722 INFO:teuthology.orchestra.run.smithi007.stdout:2202 read 0x19bf7ba thru 0x19c7a19 (0x8260 bytes) 2024-09-12T09:30:42.722 INFO:teuthology.orchestra.run.smithi007.stdout:2203 write 0x6957e5c thru 0x6961e06 (0x9fab bytes) 2024-09-12T09:30:42.728 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:42.729 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:42.729 INFO:teuthology.orchestra.run.smithi007.stdout:2205 writesame 0x22d5e6a thru 0x22df57b data_size 0x2(0x9712 bytes) 2024-09-12T09:30:42.746 INFO:teuthology.orchestra.run.smithi007.stdout:2206 punch from 0xe23bfe9 to 0xe242dd4, (0x6deb bytes) 2024-09-12T09:30:42.746 INFO:teuthology.orchestra.run.smithi007.stdout:2209 write 0xba8fe47 thru 0xba97433 (0x75ed bytes) 2024-09-12T09:30:42.753 INFO:teuthology.orchestra.run.smithi007.stdout:2210 read 0x79713ed thru 0x797ae29 (0x9a3d bytes) 2024-09-12T09:30:42.754 INFO:teuthology.orchestra.run.smithi007.stdout:2211 writesame 0x4d9cd9 thru 0x4dd408 data_size 0x8(0x3730 bytes) 2024-09-12T09:30:42.761 INFO:teuthology.orchestra.run.smithi007.stdout:2212 read 0x52eb514 thru 0x52f232b (0x6e18 bytes) 2024-09-12T09:30:42.762 INFO:teuthology.orchestra.run.smithi007.stdout:2213 write 0x462d707 thru 0x462f6dc (0x1fd6 bytes) 2024-09-12T09:30:42.768 INFO:teuthology.orchestra.run.smithi007.stdout:2214 punch from 0x78138 to 0x8418d, (0xc055 bytes) 2024-09-12T09:30:42.780 INFO:teuthology.orchestra.run.smithi007.stdout:2215 trunc from 0xe260282 to 0x81c8249 2024-09-12T09:30:42.807 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:42.807+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:42.825 INFO:teuthology.orchestra.run.smithi007.stdout:2217 write 0x9088740 thru 0x90887a3 (0x64 bytes) 2024-09-12T09:30:42.845 INFO:teuthology.orchestra.run.smithi007.stdout:2219 trunc from 0x90887a4 to 0xd5f3802 2024-09-12T09:30:42.855 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.4571313, 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:30:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:30:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.457071890000002, 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.457015304000002, 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456946991000002, 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456888804999998, 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456828272000003, 2024-09-12T09:30:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456768271999998, 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456711923, 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456652613000003, 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.45659698, 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:42.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456540597, 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:42.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456484027000002, 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456427611000002, 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:42.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:42.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456372117999997, 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:42.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:42.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456284105000002, 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:42.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:42.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.456219662000002, 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:42.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:42.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.609771876000003, 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:42.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:42.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.426355690999998, 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:42.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.310289588000003, 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:42.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:42.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:42.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:42.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:42.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.197259729999999, 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:42.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:42.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:42.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:42.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:42.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:42.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:42.962 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:42.964 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:42.959+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:42.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:42.965 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2221 read 0x797b979 thru 0x798308e (0x7716 bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2223 writesame 0xe78e8c6 thru 0xe79a172 data_size 0x67(0xb8ad bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2225 writesame 0xc74c430 thru 0xc75439a data_size 0x53(0x7f6b bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2226 write 0x1ac585 thru 0x1b42c9 (0x7d45 bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2228 writesame 0x748d021 thru 0x749a8a7 data_size 0x9(0xd887 bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2229 write 0x8aa1ba4 thru 0x8aa2f59 (0x13b6 bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2230 read 0x7afb42d thru 0x7afb8cc (0x4a0 bytes) 2024-09-12T09:30:42.966 INFO:teuthology.orchestra.run.smithi007.stdout:2231 read 0x900f1f0 thru 0x9019330 (0xa141 bytes) 2024-09-12T09:30:42.967 INFO:teuthology.orchestra.run.smithi007.stdout:2233 clone 24 order 25 su 1048576 sc 13 2024-09-12T09:30:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.1. 2024-09-12T09:30:43.064 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:43.064+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:43.088 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:43.088 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:43.089 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:43.089 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:43.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:43.165 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:43.163+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:43.266 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:43.263+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:43.273 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:43.273 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:43.273 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:43.273 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:43.274 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:43.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:43.367 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:43.364+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.279198637, 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:30:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:30:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:30:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177863094999999, 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177837642999997, 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:30:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177812791000001, 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:30:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:30:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:30:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177787453000001, 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:30:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177756422999998, 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:30:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:30:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:30:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177731262000002, 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:30:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177705361999998, 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:30:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:30:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177681755999998, 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:30:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177657244000002, 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177632805999998, 2024-09-12T09:30:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177608219, 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:30:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177584203000002, 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:30:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177556361000001, 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:30:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177531516000002, 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177506790000002, 2024-09-12T09:30:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:30:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:30:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.177482292999997, 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.171676421999997, 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:30:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:30:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.026724457999997, 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:43.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:43.517 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.20922766, 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:30:43.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:43.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:43.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:43.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:43.523 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:43.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:43.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:43.526 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:43.523+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:43.625 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:43.623+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:43.693 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:43.693 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:43.693 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:43.693 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:43.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:43.729 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:43.729 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:43.728+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:43.746 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:43.763 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:43.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:43.780 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:43.801 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:43.801 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:43.821 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:43.821 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:43.827 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:43.824+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:43.828 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:43.828 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:43.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:43.843 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:43.865 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:43.932 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:43.932+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:43.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:43.969 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:43.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:44.032 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:44.032+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:44.129 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:44.128+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:44.168 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:44.168 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:44.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:44.268 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:44.268+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:44.331 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:44.328+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:44.344 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:44.344 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:44.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:44.448 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:44.448+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:44.525 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:44.525 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:44.534 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:44.531+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:44.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:44.635 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:44.631+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:44.713 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:44.713 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:44.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:44.735 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:44.731+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:44.836 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:44.835+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:44.855 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:44.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:44.937 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:44.935+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:30:45.004 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.005 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:45.072 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:45.071+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:45.078 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone23 from 0xe79a173 (overlap 0x81c8249) to 0x74cdba8 2024-09-12T09:30:45.139 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:45.135+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:45.194 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.195 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:45.256 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:45.255+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:45.273 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-12T09:30:45.340 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:45.339+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:45.380 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.380 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:45.444 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:45.443+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:45.543 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:45.539+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:45.568 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.568 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:45.643 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:45.640+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:45.729 INFO:teuthology.orchestra.run.smithi007.stdout:2234 write 0x58d28d2 thru 0x58db65a (0x8d89 bytes) 2024-09-12T09:30:45.744 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:45.743+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:45.749 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.749 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:45.787 INFO:teuthology.orchestra.run.smithi007.stdout:2235 read 0xc540d2f thru 0xc54e24e (0xd520 bytes) 2024-09-12T09:30:45.787 INFO:teuthology.orchestra.run.smithi007.stdout:2236 read 0x6bb59ae thru 0x6bc3c33 (0xe286 bytes) 2024-09-12T09:30:45.788 INFO:teuthology.orchestra.run.smithi007.stdout:2237 read 0x595dba5 thru 0x59655a6 (0x7a02 bytes) 2024-09-12T09:30:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:2238 write 0x30cfa1b thru 0x30cffdb (0x5c1 bytes) 2024-09-12T09:30:45.845 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:45.844+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:45.865 INFO:teuthology.orchestra.run.smithi007.stdout:2239 trunc from 0xe79a173 to 0x29c7abc 2024-09-12T09:30:45.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:45.945 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:45.943+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:45.952 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:45.952 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:45.959 INFO:teuthology.orchestra.run.smithi007.stdout:2240 trunc from 0x29c7abc to 0x81a1bb1 2024-09-12T09:30:45.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:46.046 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:46.044+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:46.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:46.078 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:46.078 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:46.094 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:46.113 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:46.133 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:46.138 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:46.138 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:46.146 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:46.144+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:46.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:46.154 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:46.154 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:46.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:46.192 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:46.248 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:46.247+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:46.276 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:46.276 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:46.276 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:46.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:46.376 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:46.375+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:46.450 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:46.447+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:46.456 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:46.456 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:46.457 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:46.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:46.551 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:46.548+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:46.637 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:46.638 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:46.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:46.638 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:46.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.639 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.639 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:46.639 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999615759000001, 2024-09-12T09:30:46.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:46.639 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:46.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:46.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:46.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:46.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:46.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:46.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:46.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:46.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:46.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999566151000003, 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999532408999997, 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:46.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:46.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999485370999999, 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999427531000002, 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:46.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:46.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999368443999998, 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:46.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:46.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.675 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999334257000001, 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:46.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999302921000002, 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:46.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:46.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:46.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999272879999999, 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:46.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:46.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999241634999997, 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:46.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999199865999998, 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:46.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999168400000002, 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:46.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999120859999998, 2024-09-12T09:30:46.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:46.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:46.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999087822999996, 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999042803000002, 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:46.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.999010145, 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.99897833, 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:46.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:46.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:46.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.998920417000001, 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.998878435000002, 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:46.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:46.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.998842453000002, 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:46.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:46.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:46.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:46.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:46.742 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:46.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:46.744 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:46.739+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:46.844 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:46.844+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:46.919 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:46.920 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:46.920 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:46.920 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:46.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:46.945 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:46.944+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:46.981 INFO:teuthology.orchestra.run.smithi007.stdout:2241 punch from 0x37d8331 to 0x37de095, (0x5d64 bytes) 2024-09-12T09:30:46.982 INFO:teuthology.orchestra.run.smithi007.stdout:2243 punch from 0x3eb1997 to 0x3eb6c40, (0x52a9 bytes) 2024-09-12T09:30:46.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:30:47.003 INFO:teuthology.orchestra.run.smithi007.stdout:2246 write 0xc840072 thru 0xc847294 (0x7223 bytes) 2024-09-12T09:30:47.006 INFO:teuthology.orchestra.run.smithi007.stdout:2247 read 0x3a3fca3 thru 0x3a4bae1 (0xbe3f bytes) 2024-09-12T09:30:47.008 INFO:teuthology.orchestra.run.smithi007.stdout:2248 punch from 0x2806e0 to 0x285a87, (0x53a7 bytes) 2024-09-12T09:30:47.008 INFO:teuthology.orchestra.run.smithi007.stdout:2249 read 0x990896a thru 0x9915144 (0xc7db bytes) 2024-09-12T09:30:47.008 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:47.008 INFO:teuthology.orchestra.run.smithi007.stdout:2251 write 0x5973aaa thru 0x5977194 (0x36eb bytes) 2024-09-12T09:30:47.011 INFO:teuthology.orchestra.run.smithi007.stdout:2253 write 0x7c6b0cc thru 0x7c6b8d2 (0x807 bytes) 2024-09-12T09:30:47.037 INFO:teuthology.orchestra.run.smithi007.stdout:2255 trunc from 0xc847295 to 0x3a639ca 2024-09-12T09:30:47.046 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:47.044+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:47.052 INFO:teuthology.orchestra.run.smithi007.stdout:2256 trunc from 0x3a639ca to 0x53d7c6f 2024-09-12T09:30:47.059 INFO:teuthology.orchestra.run.smithi007.stdout:2257 read 0x143cf9b thru 0x1447e48 (0xaeae bytes) 2024-09-12T09:30:47.060 INFO:teuthology.orchestra.run.smithi007.stdout:2258 read 0x1be6fa thru 0x1c1aed (0x33f4 bytes) 2024-09-12T09:30:47.063 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:47.063 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:47.063 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:47.063 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:47.063 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:47.064 INFO:teuthology.orchestra.run.smithi007.stdout:2259 trunc from 0x53d7c6f to 0x3c2a39c 2024-09-12T09:30:47.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:47.082 INFO:teuthology.orchestra.run.smithi007.stdout:2260 write 0x9506d7 thru 0x9528e7 (0x2211 bytes) 2024-09-12T09:30:47.083 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:47.083 INFO:teuthology.orchestra.run.smithi007.stdout:2262 write 0x19cd8da thru 0x19da9e2 (0xd109 bytes) 2024-09-12T09:30:47.085 INFO:teuthology.orchestra.run.smithi007.stdout:2264 punch from 0x308ad15 to 0x308b23b, (0x526 bytes) 2024-09-12T09:30:47.107 INFO:teuthology.orchestra.run.smithi007.stdout:2265 writesame 0xad86cf4 thru 0xad88fcc data_size 0x1(0x22d9 bytes) 2024-09-12T09:30:47.109 INFO:teuthology.orchestra.run.smithi007.stdout:2266 write 0x12c61f0 thru 0x12cc64c (0x645d bytes) 2024-09-12T09:30:47.113 INFO:teuthology.orchestra.run.smithi007.stdout:2268 write 0x385ae47 thru 0x3868495 (0xd64f bytes) 2024-09-12T09:30:47.116 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:47.128 INFO:teuthology.orchestra.run.smithi007.stdout:2270 write 0xd0adc2f thru 0xd0b35fc (0x59ce bytes) 2024-09-12T09:30:47.130 INFO:teuthology.orchestra.run.smithi007.stdout:2271 write 0x5d8b00f thru 0x5d992f8 (0xe2ea bytes) 2024-09-12T09:30:47.132 INFO:teuthology.orchestra.run.smithi007.stdout:2272 clone 25 order 21 su 262144 sc 14 2024-09-12T09:30:47.146 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:47.144+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:47.208 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809350432000002, 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:47.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809291021999996, 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:47.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809234435999997, 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:47.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:47.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809166122999997, 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809107937, 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:47.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:47.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.809047403999998, 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.232 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:47.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:47.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808987404, 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:47.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808931055000002, 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:47.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:47.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808871744999998, 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:47.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808816112000002, 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:47.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:47.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:47.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808759729000002, 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:47.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:47.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808703158999997, 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:47.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:47.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808646742999997, 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.270 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:47.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808591249999999, 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:47.275 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.280 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808503236999996, 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:47.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.808438793999997, 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.961991007999998, 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:47.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:47.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.778574823, 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:47.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.662508719999998, 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.549478862000001, 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:47.315 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:47.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:47.317 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:47.316+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:47.417 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:47.416+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:47.452 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:47.452 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:47.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:47.519 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:47.515+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:47.620 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:47.619+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:47.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:47.633 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:47.633 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:47.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:47.721 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:47.719+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:47.770 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.630797121999997, 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:30:47.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:30:47.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:30:47.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:47.778 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529461580000003, 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:30:47.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:47.783 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529436128, 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:30:47.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:30:47.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529411275999998, 2024-09-12T09:30:47.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:30:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529385937999997, 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:30:47.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529354908000002, 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:30:47.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529329746999998, 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:30:47.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:30:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529303847000001, 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:47.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:30:47.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529280241000002, 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:30:47.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:30:47.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529255728999999, 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:47.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:30:47.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529231291000002, 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:30:47.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529206704000003, 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:30:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529182687999999, 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:30:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:30:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529154845999997, 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:30:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:30:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529130000999999, 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529105274999999, 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:30:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.529080778000001, 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:30:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.523274907000001, 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:30:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.378322943000001, 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:30:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.560826145, 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:30:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:30:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:47.875 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:47.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:47.877 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:47.876+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:30:47.977 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:47.975+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:47.978 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 0, 7] out_osds: [6, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:30:47.978 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T09:30:48.063 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:48.063 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:48.063 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:48.078 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:48.075+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:48.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:48.127 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone24 from 0xd0b35fd (overlap 0x29c7abc) to 0xc88b38 2024-09-12T09:30:48.178 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:48.175+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:48.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:48.218 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:48.218 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:48.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:48.278 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:48.275+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-12T09:30:48.382 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:48.379+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:48.400 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.214005230000001, 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:48.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:48.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:30:48.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:30:48.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.199034501, 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.882187497, 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:30:48.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.430 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.882141138, 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.882033917999999, 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:30:48.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881940427, 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:30:48.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:30:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881873914, 2024-09-12T09:30:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:30:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881813104999999, 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881753813, 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:30:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881693304000001, 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:30:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881615248999999, 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:30:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881529874, 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:30:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:30:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881461555, 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:30:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:30:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:30:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.88136897, 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881313061, 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881280737999999, 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:30:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:30:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881247145, 2024-09-12T09:30:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:30:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881196203, 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:30:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:30:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881098048, 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:30:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.881040953999999, 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:30:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:48.513 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:48.514 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:48.515 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:48.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:48.516 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:48.516 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:48.516 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:48.517 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:48.517 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:48.518 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:48.511+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:48.569 INFO:teuthology.orchestra.run.smithi007.stdout:2273 read 0xbde5513 thru 0xbde7987 (0x2475 bytes) 2024-09-12T09:30:48.571 INFO:teuthology.orchestra.run.smithi007.stdout:2274 punch from 0x490e7cc to 0x491b0ab, (0xc8df bytes) 2024-09-12T09:30:48.577 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:48.577 INFO:teuthology.orchestra.run.smithi007.stdout:2276 write 0x8db15d0 thru 0x8dbe552 (0xcf83 bytes) 2024-09-12T09:30:48.598 INFO:teuthology.orchestra.run.smithi007.stdout:2278 write 0xe2fc180 thru 0xe304f91 (0x8e12 bytes) 2024-09-12T09:30:48.613 INFO:teuthology.orchestra.run.smithi007.stdout:2279 trunc from 0xe304f92 to 0x83023d6 2024-09-12T09:30:48.616 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:48.615+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:48.659 INFO:teuthology.orchestra.run.smithi007.stdout:2281 write 0x17b1906 thru 0x17c094f (0xf04a bytes) 2024-09-12T09:30:48.676 INFO:teuthology.orchestra.run.smithi007.stdout:2284 writesame 0x31ff903 thru 0x320f373 data_size 0x81(0xfa71 bytes) 2024-09-12T09:30:48.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:48.686 INFO:teuthology.orchestra.run.smithi007.stdout:2287 punch from 0x4cfb196 to 0x4d03866, (0x86d0 bytes) 2024-09-12T09:30:48.687 INFO:teuthology.orchestra.run.smithi007.stdout:2288 writesame 0x438c1e9 thru 0x439ae12 data_size 0x2b(0xec2a bytes) 2024-09-12T09:30:48.692 INFO:teuthology.orchestra.run.smithi007.stdout:2289 read 0x70501ed thru 0x7051aed (0x1901 bytes) 2024-09-12T09:30:48.711 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:48.711 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:48.715 INFO:teuthology.orchestra.run.smithi007.stdout:2290 write 0xdb511e7 thru 0xdb5329b (0x20b5 bytes) 2024-09-12T09:30:48.716 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:48.715+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:48.718 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:48.718 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:48.719 INFO:teuthology.orchestra.run.smithi007.stdout:2291 read 0x68423a5 thru 0x684fbfd (0xd859 bytes) 2024-09-12T09:30:48.721 INFO:teuthology.orchestra.run.smithi007.stdout:2293 read 0x30a2d21 thru 0x30ad1a3 (0xa483 bytes) 2024-09-12T09:30:48.721 INFO:teuthology.orchestra.run.smithi007.stdout:2295 write 0xb2cf76c thru 0xb2d2458 (0x2ced bytes) 2024-09-12T09:30:48.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:48.740 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:48.744 INFO:teuthology.orchestra.run.smithi007.stdout:2296 trunc from 0xdb5329c to 0x3f527e0 2024-09-12T09:30:48.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:48.777 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:48.788 INFO:teuthology.orchestra.run.smithi007.stdout:2297 write 0x25e9d80 thru 0x25f57f8 (0xba79 bytes) 2024-09-12T09:30:48.795 INFO:teuthology.orchestra.run.smithi007.stdout:2298 write 0x95479c thru 0x964704 (0xff69 bytes) 2024-09-12T09:30:48.821 INFO:teuthology.orchestra.run.smithi007.stdout:2299 trunc from 0x3f527e0 to 0xebc9da9 2024-09-12T09:30:48.821 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:48.820+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:2300 punch from 0xb29c180 to 0xb2a2e45, (0x6cc5 bytes) 2024-09-12T09:30:48.832 INFO:teuthology.orchestra.run.smithi007.stdout:2301 writesame 0xb8c4246 thru 0xb8cbd89 data_size 0x31(0x7b44 bytes) 2024-09-12T09:30:48.837 INFO:teuthology.orchestra.run.smithi007.stdout:2302 write 0xeb343d3 thru 0xeb4270b (0xe339 bytes) 2024-09-12T09:30:48.864 INFO:teuthology.orchestra.run.smithi007.stdout:2303 trunc from 0xebc9da9 to 0xa5fea4 2024-09-12T09:30:48.917 INFO:teuthology.orchestra.run.smithi007.stdout:2304 punch from 0x43dcdb to 0x4463d0, (0x86f5 bytes) 2024-09-12T09:30:48.917 INFO:teuthology.orchestra.run.smithi007.stdout:2308 punch from 0x53b2ac to 0x54a857, (0xf5ab bytes) 2024-09-12T09:30:48.917 INFO:teuthology.orchestra.run.smithi007.stdout:2309 punch from 0x7fe577 to 0x803ed7, (0x5960 bytes) 2024-09-12T09:30:48.917 INFO:teuthology.orchestra.run.smithi007.stdout:2310 read 0x65c138 thru 0x66b2e6 (0xf1af bytes) 2024-09-12T09:30:48.918 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:48.916+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:48.918 INFO:teuthology.orchestra.run.smithi007.stdout:2311 punch from 0x5e8b33 to 0x5f557e, (0xca4b bytes) 2024-09-12T09:30:48.918 INFO:teuthology.orchestra.run.smithi007.stdout:2312 punch from 0x6bb291 to 0x6bfe83, (0x4bf2 bytes) 2024-09-12T09:30:48.920 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:48.920 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:48.932 INFO:teuthology.orchestra.run.smithi007.stdout:2313 writesame 0x2d03aaf thru 0x2d12ef7 data_size 0x1(0xf449 bytes) 2024-09-12T09:30:48.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:48.952 INFO:teuthology.orchestra.run.smithi007.stdout:2314 write 0x7424370 thru 0x742b5bb (0x724c bytes) 2024-09-12T09:30:48.957 INFO:teuthology.orchestra.run.smithi007.stdout:2316 read 0xa0587c thru 0xa12ba6 (0xd32b bytes) 2024-09-12T09:30:48.981 INFO:teuthology.orchestra.run.smithi007.stdout:2317 write 0xde1fbb4 thru 0xde2c2e5 (0xc732 bytes) 2024-09-12T09:30:48.988 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T09:30:48.996 INFO:teuthology.orchestra.run.smithi007.stdout:2319 punch from 0xdd52cb6 to 0xdd58ba1, (0x5eeb bytes) 2024-09-12T09:30:48.996 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:48.997 INFO:teuthology.orchestra.run.smithi007.stdout:2321 write 0x9b70c34 thru 0x9b7b746 (0xab13 bytes) 2024-09-12T09:30:49.002 INFO:teuthology.orchestra.run.smithi007.stdout:2322 write 0x738559d thru 0x739147f (0xbee3 bytes) 2024-09-12T09:30:49.006 INFO:teuthology.orchestra.run.smithi007.stdout:2323 writesame 0xaedd964 thru 0xaee2d25 data_size 0x2(0x53c2 bytes) 2024-09-12T09:30:49.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T09:30:49.023 INFO:teuthology.orchestra.run.smithi007.stdout:2324 write 0x9efa042 thru 0x9f0282d (0x87ec bytes) 2024-09-12T09:30:49.023 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:49.020+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:49.027 INFO:teuthology.orchestra.run.smithi007.stdout:2325 punch from 0xd632356 to 0xd63c477, (0xa121 bytes) 2024-09-12T09:30:49.039 INFO:teuthology.orchestra.run.smithi007.stdout:2326 write 0xdf88066 thru 0xdf927ec (0xa787 bytes) 2024-09-12T09:30:49.044 INFO:teuthology.orchestra.run.smithi007.stdout:2328 write 0xe1a63a thru 0xe1fc77 (0x563e bytes) 2024-09-12T09:30:49.047 INFO:teuthology.orchestra.run.smithi007.stdout:2330 read 0x3471b9e thru 0x3477833 (0x5c96 bytes) 2024-09-12T09:30:49.047 INFO:teuthology.orchestra.run.smithi007.stdout:2331 read 0xbcec391 thru 0xbcf511f (0x8d8f bytes) 2024-09-12T09:30:49.048 INFO:teuthology.orchestra.run.smithi007.stdout:2332 read 0x3582109 thru 0x3590e4a (0xed42 bytes) 2024-09-12T09:30:49.048 INFO:teuthology.orchestra.run.smithi007.stdout:2333 writesame 0xa5a902a thru 0xa5af52f data_size 0x43(0x6506 bytes) 2024-09-12T09:30:49.054 INFO:teuthology.orchestra.run.smithi007.stdout:2334 writesame 0xa3372e3 thru 0xa3424f9 data_size 0x5b(0xb217 bytes) 2024-09-12T09:30:49.060 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.060 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.061 INFO:teuthology.orchestra.run.smithi007.stdout:2338 trunc from 0xdf927ed to 0xd48e801 2024-09-12T09:30:49.075 INFO:teuthology.orchestra.run.smithi007.stdout:2339 read 0xae0b8b7 thru 0xae1005a (0x47a4 bytes) 2024-09-12T09:30:49.075 INFO:teuthology.orchestra.run.smithi007.stdout:2340 write 0x9e89785 thru 0x9e96530 (0xcdac bytes) 2024-09-12T09:30:49.079 INFO:teuthology.orchestra.run.smithi007.stdout:2341 read 0x9263ea7 thru 0x9269274 (0x53ce bytes) 2024-09-12T09:30:49.079 INFO:teuthology.orchestra.run.smithi007.stdout:2343 read 0x9d457e1 thru 0x9d53288 (0xdaa8 bytes) 2024-09-12T09:30:49.080 INFO:teuthology.orchestra.run.smithi007.stdout:2344 write 0x1f3a8ec thru 0x1f43b03 (0x9218 bytes) 2024-09-12T09:30:49.085 INFO:teuthology.orchestra.run.smithi007.stdout:2346 read 0xb121f35 thru 0xb12adc6 (0x8e92 bytes) 2024-09-12T09:30:49.085 INFO:teuthology.orchestra.run.smithi007.stdout:2348 write 0xcb1bb9a thru 0xcb2b0fd (0xf564 bytes) 2024-09-12T09:30:49.090 INFO:teuthology.orchestra.run.smithi007.stdout:2349 write 0x101d3b thru 0x106071 (0x4337 bytes) 2024-09-12T09:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.093 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.097 INFO:teuthology.orchestra.run.smithi007.stdout:2352 trunc from 0xd48e801 to 0xb061026 2024-09-12T09:30:49.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:2353 read 0x3fb5350 thru 0x3fb933a (0x3feb bytes) 2024-09-12T09:30:49.119 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:49.116+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout:2354 trunc from 0xb061026 to 0x504b259 2024-09-12T09:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout:2356 punch from 0x5014cdb to 0x5020c0f, (0xbf34 bytes) 2024-09-12T09:30:49.149 INFO:teuthology.orchestra.run.smithi007.stdout:2357 trunc from 0x504b259 to 0x54a66d5 2024-09-12T09:30:49.159 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.173 INFO:teuthology.orchestra.run.smithi007.stdout:2359 write 0x78f269e thru 0x78f5467 (0x2dca bytes) 2024-09-12T09:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout:2363 read 0x5da958d thru 0x5db4ad1 (0xb545 bytes) 2024-09-12T09:30:49.178 INFO:teuthology.orchestra.run.smithi007.stdout:2364 writesame 0x27393d1 thru 0x27413dc data_size 0x37(0x800c bytes) 2024-09-12T09:30:49.183 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.197 INFO:teuthology.orchestra.run.smithi007.stdout:2367 write 0xa44b45a thru 0xa44c1c8 (0xd6f bytes) 2024-09-12T09:30:49.202 INFO:teuthology.orchestra.run.smithi007.stdout:2368 writesame 0x91e9abc thru 0x91f4bba data_size 0x1(0xb0ff bytes) 2024-09-12T09:30:49.221 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:49.219+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:49.260 INFO:teuthology.orchestra.run.smithi007.stdout:2369 write 0x18b885c thru 0x18be4e6 (0x5c8b bytes) 2024-09-12T09:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:2371 read 0x46f878c thru 0x46fe005 (0x587a bytes) 2024-09-12T09:30:49.264 INFO:teuthology.orchestra.run.smithi007.stdout:2372 writesame 0x650c1f0 thru 0x65122ec data_size 0x1(0x60fd bytes) 2024-09-12T09:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.272 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:2373 punch from 0x41f1284 to 0x41fba15, (0xa791 bytes) 2024-09-12T09:30:49.307 INFO:teuthology.orchestra.run.smithi007.stdout:2374 trunc from 0xa44c1c9 to 0xdaabb37 2024-09-12T09:30:49.319 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.319 INFO:teuthology.orchestra.run.smithi007.stdout:2376 read 0x58cc6be thru 0x58da776 (0xe0b9 bytes) 2024-09-12T09:30:49.321 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:49.320+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:49.331 INFO:teuthology.orchestra.run.smithi007.stdout:2377 write 0xdeb1334 thru 0xdebb2ae (0x9f7b bytes) 2024-09-12T09:30:49.337 INFO:teuthology.orchestra.run.smithi007.stdout:2378 punch from 0x5b6bc5f to 0x5b723f9, (0x679a bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2380 read 0x580598e thru 0x58094cf (0x3b42 bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2381 read 0x69b3258 thru 0x69be584 (0xb32d bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2383 punch from 0x7436a36 to 0x74416bd, (0xac87 bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2384 punch from 0x5a74a9b to 0x5a7fc8e, (0xb1f3 bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2385 read 0x40c3ec2 thru 0x40c78ca (0x3a09 bytes) 2024-09-12T09:30:49.338 INFO:teuthology.orchestra.run.smithi007.stdout:2386 writesame 0x833f27b thru 0x8347611 data_size 0x39(0x8397 bytes) 2024-09-12T09:30:49.342 INFO:teuthology.orchestra.run.smithi007.stdout:2387 writesame 0x6f220d4 thru 0x6f2c4e2 data_size 0x1(0xa40f bytes) 2024-09-12T09:30:49.395 INFO:teuthology.orchestra.run.smithi007.stdout:2389 read 0x468b71e thru 0x469a070 (0xe953 bytes) 2024-09-12T09:30:49.395 INFO:teuthology.orchestra.run.smithi007.stdout:2392 read 0xbb600b4 thru 0xbb683fd (0x834a bytes) 2024-09-12T09:30:49.395 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.395 INFO:teuthology.orchestra.run.smithi007.stdout:2395 read 0xd7bf540 thru 0xd7cca85 (0xd546 bytes) 2024-09-12T09:30:49.396 INFO:teuthology.orchestra.run.smithi007.stdout:2397 writesame 0x93db498 thru 0x93e58b1 data_size 0x5(0xa41a bytes) 2024-09-12T09:30:49.408 INFO:teuthology.orchestra.run.smithi007.stdout:2398 punch from 0x15dd5e1 to 0x15e89d4, (0xb3f3 bytes) 2024-09-12T09:30:49.408 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.408 INFO:teuthology.orchestra.run.smithi007.stdout:2400 write 0x8bc7def thru 0x8bc8aaa (0xcbc bytes) 2024-09-12T09:30:49.413 INFO:teuthology.orchestra.run.smithi007.stdout:2401 write 0x1be0dc1 thru 0x1be9101 (0x8341 bytes) 2024-09-12T09:30:49.417 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.417 INFO:teuthology.orchestra.run.smithi007.stdout:2403 write 0xaf5586a thru 0xaf5a8bc (0x5053 bytes) 2024-09-12T09:30:49.422 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:49.420+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:49.422 INFO:teuthology.orchestra.run.smithi007.stdout:2404 write 0x376db10 thru 0x3776b2e (0x901f bytes) 2024-09-12T09:30:49.427 INFO:teuthology.orchestra.run.smithi007.stdout:2405 punch from 0xb84a523 to 0xb853103, (0x8be0 bytes) 2024-09-12T09:30:49.427 INFO:teuthology.orchestra.run.smithi007.stdout:2407 punch from 0x81f05b8 to 0x81f7258, (0x6ca0 bytes) 2024-09-12T09:30:49.427 INFO:teuthology.orchestra.run.smithi007.stdout:2408 writesame 0xa09838a thru 0xa0a1c21 data_size 0x4c(0x9898 bytes) 2024-09-12T09:30:49.432 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.432 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.433 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.449 INFO:teuthology.orchestra.run.smithi007.stdout:2412 trunc from 0xdebb2af to 0x314e6e5 2024-09-12T09:30:49.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:49.475 INFO:teuthology.orchestra.run.smithi007.stdout:2413 read 0x1cf7f49 thru 0x1d03f70 (0xc028 bytes) 2024-09-12T09:30:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:2415 read 0x1662ca7 thru 0x166b56f (0x88c9 bytes) 2024-09-12T09:30:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:2416 punch from 0xb157df to 0xb16a46, (0x1267 bytes) 2024-09-12T09:30:49.477 INFO:teuthology.orchestra.run.smithi007.stdout:2418 read 0x13bdacc thru 0x13c450c (0x6a41 bytes) 2024-09-12T09:30:49.507 INFO:teuthology.orchestra.run.smithi007.stdout:2421 write 0xcd34c5a thru 0xcd36041 (0x13e8 bytes) 2024-09-12T09:30:49.512 INFO:teuthology.orchestra.run.smithi007.stdout:2422 writesame 0x1aa2bac thru 0x1ab08dd data_size 0x17(0xdd32 bytes) 2024-09-12T09:30:49.515 INFO:teuthology.orchestra.run.smithi007.stdout:2424 clone 26 order 21 su 262144 sc 7 2024-09-12T09:30:49.523 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:49.520+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:49.596 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.596 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:49.656 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:49.655+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:49.725 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:49.724+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:49.742 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.742 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:49.844 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:49.843+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:49.917 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:49.917 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:49.926 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:49.924+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:49.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T09:30:50.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:30:50.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T09:30:50.028 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:50.024+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:50.105 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.105 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:50.136 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone25 from 0xcd36042 (overlap 0xa5fea4) to 0x2132ed 2024-09-12T09:30:50.168 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:50.167+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:50.229 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:50.227+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-12T09:30:50.269 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:50.304 INFO:teuthology.orchestra.run.smithi007.stdout:2425 read 0xc615b0b thru 0xc61cb6d (0x7063 bytes) 2024-09-12T09:30:50.305 INFO:teuthology.orchestra.run.smithi007.stdout:2426 punch from 0x4d38296 to 0x4d43ebc, (0xbc26 bytes) 2024-09-12T09:30:50.311 INFO:teuthology.orchestra.run.smithi007.stdout:2428 write 0x9c61cbd thru 0x9c67342 (0x5686 bytes) 2024-09-12T09:30:50.316 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:50.316 INFO:teuthology.orchestra.run.smithi007.stdout:2431 write 0x134e99 thru 0x135e4a (0xfb2 bytes) 2024-09-12T09:30:50.325 INFO:teuthology.orchestra.run.smithi007.stdout:2432 write 0x633075d thru 0x633d644 (0xcee8 bytes) 2024-09-12T09:30:50.331 INFO:teuthology.orchestra.run.smithi007.stdout:2433 read 0x17ac967 thru 0x17b90f9 (0xc793 bytes) 2024-09-12T09:30:50.341 INFO:teuthology.orchestra.run.smithi007.stdout:2434 trunc from 0xcd36042 to 0x852c3e5 2024-09-12T09:30:50.356 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:50.355+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:50.366 INFO:teuthology.orchestra.run.smithi007.stdout:2435 punch from 0x4abd886 to 0x4ac9473, (0xbbed bytes) 2024-09-12T09:30:50.366 INFO:teuthology.orchestra.run.smithi007.stdout:2436 writesame 0x2afdb68 thru 0x2b083c5 data_size 0x2e(0xa85e bytes) 2024-09-12T09:30:50.377 INFO:teuthology.orchestra.run.smithi007.stdout:2437 trunc from 0x852c3e5 to 0xa094e7c 2024-09-12T09:30:50.385 INFO:teuthology.orchestra.run.smithi007.stdout:2438 read 0x194aa5a thru 0x1951936 (0x6edd bytes) 2024-09-12T09:30:50.386 INFO:teuthology.orchestra.run.smithi007.stdout:2439 read 0x7eef2bc thru 0x7efc2a4 (0xcfe9 bytes) 2024-09-12T09:30:50.387 INFO:teuthology.orchestra.run.smithi007.stdout:2441 punch from 0x581591f to 0x5823db4, (0xe495 bytes) 2024-09-12T09:30:50.387 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:50.387 INFO:teuthology.orchestra.run.smithi007.stdout:2443 read 0x2e0ddb0 thru 0x2e18f1a (0xb16b bytes) 2024-09-12T09:30:50.404 INFO:teuthology.orchestra.run.smithi007.stdout:2444 trunc from 0xa094e7c to 0x422825 2024-09-12T09:30:50.431 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:50.427+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:50.443 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:50.443 INFO:teuthology.orchestra.run.smithi007.stdout:2447 clone 27 order 25 su 524288 sc 2 2024-09-12T09:30:50.451 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:50.532 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:50.532+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:50.629 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.637 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.639 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:50.635+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:50.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:50.740 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:50.736+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:50.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:50.814 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.814 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:50.848 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:50.848+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:50.941 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:50.939+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:50.945 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:50.945 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:50.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:50.960 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:50.960 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:50.976 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:50.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:50.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:51.010 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:51.010 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:51.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:51.042 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:51.040+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:30:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T09:30:51.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:51.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T09:30:51.075 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:51.075 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:51.096 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503450595999993, 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:30:51.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:30:51.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503400987999996, 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:30:51.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:30:51.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.152 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503367245999996, 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.153 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:30:51.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.157 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503320208000005, 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:30:51.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.162 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503262367999994, 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503203280999998, 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:30:51.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503169094, 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:30:51.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.176 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503137757999994, 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:30:51.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503107717000006, 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.181 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:30:51.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503076472000004, 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:30:51.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503034702999997, 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:30:51.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:30:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.503003237000001, 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:30:51.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:30:51.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502955697000004, 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:30:51.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502922659999996, 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:30:51.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:30:51.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.210 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502877639999994, 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:30:51.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:30:51.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:51.215 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502844981999999, 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:30:51.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502813166999999, 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:30:51.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502755253999993, 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:30:51.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502713271999994, 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:30:51.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:30:51.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.502677289999994, 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.240 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:51.240 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:51.160+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:51.241 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone26 from 0x422825 (overlap 0x422825) to 0x1a7675 2024-09-12T09:30:51.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:51.242 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:51.240+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:51.343 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:51.340+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:51.349 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-12T09:30:51.374 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:51.374 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:51.374 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:30:51.374 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:51.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:51.396 INFO:teuthology.orchestra.run.smithi007.stdout:2449 write 0xa7ca86e thru 0xa7d63d8 (0xbb6b bytes) 2024-09-12T09:30:51.401 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.401 INFO:teuthology.orchestra.run.smithi007.stdout:2452 write 0x79dd974 thru 0x79e3398 (0x5a25 bytes) 2024-09-12T09:30:51.423 INFO:teuthology.orchestra.run.smithi007.stdout:2454 write 0xdbcd3c6 thru 0xdbd141e (0x4059 bytes) 2024-09-12T09:30:51.429 INFO:teuthology.orchestra.run.smithi007.stdout:2457 punch from 0x4444606 to 0x4444add, (0x4d7 bytes) 2024-09-12T09:30:51.429 INFO:teuthology.orchestra.run.smithi007.stdout:2458 read 0xbbf16eb thru 0xbbfa961 (0x9277 bytes) 2024-09-12T09:30:51.444 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:51.443+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:51.448 INFO:teuthology.orchestra.run.smithi007.stdout:2459 trunc from 0xdbd141f to 0x4074720 2024-09-12T09:30:51.473 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.473 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.493 INFO:teuthology.orchestra.run.smithi007.stdout:2465 write 0x72a44f8 thru 0x72aa64d (0x6156 bytes) 2024-09-12T09:30:51.500 INFO:teuthology.orchestra.run.smithi007.stdout:2466 read 0x1ff7568 thru 0x1ff857e (0x1017 bytes) 2024-09-12T09:30:51.500 INFO:teuthology.orchestra.run.smithi007.stdout:2467 write 0xc8810b thru 0xc8a062 (0x1f58 bytes) 2024-09-12T09:30:51.515 INFO:teuthology.orchestra.run.smithi007.stdout:2468 writesame 0x413c736 thru 0x4141807 data_size 0x5(0x50d2 bytes) 2024-09-12T09:30:51.516 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:51.516 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:51.516 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:51.516 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:51.516 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:51.526 INFO:teuthology.orchestra.run.smithi007.stdout:2470 writesame 0x26b3e8c thru 0x26c0512 data_size 0x3(0xc687 bytes) 2024-09-12T09:30:51.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:51.556 INFO:teuthology.orchestra.run.smithi007.stdout:2471 trunc from 0x72aa64e to 0xac1ff6f 2024-09-12T09:30:51.570 INFO:teuthology.orchestra.run.smithi007.stdout:2472 read 0x1e8b7ba thru 0x1e9792b (0xc172 bytes) 2024-09-12T09:30:51.571 INFO:teuthology.orchestra.run.smithi007.stdout:2473 read 0x7f4ac2b thru 0x7f584d2 (0xd8a8 bytes) 2024-09-12T09:30:51.572 INFO:teuthology.orchestra.run.smithi007.stdout:2475 read 0x6354a27 thru 0x635c81d (0x7df7 bytes) 2024-09-12T09:30:51.573 INFO:teuthology.orchestra.run.smithi007.stdout:2476 punch from 0x1351ed8 to 0x1355aad, (0x3bd5 bytes) 2024-09-12T09:30:51.573 INFO:teuthology.orchestra.run.smithi007.stdout:2479 read 0x1f8d1c0 thru 0x1f901a2 (0x2fe3 bytes) 2024-09-12T09:30:51.576 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:51.575+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:51.593 INFO:teuthology.orchestra.run.smithi007.stdout:2480 writesame 0xcffaf6e thru 0xd007a32 data_size 0x79(0xcac5 bytes) 2024-09-12T09:30:51.594 INFO:teuthology.orchestra.run.smithi007.stdout:2481 write 0x7be5675 thru 0x7bebf00 (0x688c bytes) 2024-09-12T09:30:51.597 INFO:teuthology.orchestra.run.smithi007.stdout:2482 write 0x395f8ee thru 0x396a4ba (0xabcd bytes) 2024-09-12T09:30:51.613 INFO:teuthology.orchestra.run.smithi007.stdout:2483 write 0x48aabcd thru 0x48b9126 (0xe55a bytes) 2024-09-12T09:30:51.617 INFO:teuthology.orchestra.run.smithi007.stdout:2486 write 0x1dd9617 thru 0x1de2e82 (0x986c bytes) 2024-09-12T09:30:51.620 INFO:teuthology.orchestra.run.smithi007.stdout:2487 read 0x216c475 thru 0x2172f04 (0x6a90 bytes) 2024-09-12T09:30:51.625 INFO:teuthology.orchestra.run.smithi007.stdout:2488 trunc from 0xd007a33 to 0xba73283 2024-09-12T09:30:51.637 INFO:teuthology.orchestra.run.smithi007.stdout:2490 writesame 0xa807a7e thru 0xa8163b4 data_size 0x7(0xe937 bytes) 2024-09-12T09:30:51.645 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:51.643+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:51.661 INFO:teuthology.orchestra.run.smithi007.stdout:2491 write 0xbf4c2c8 thru 0xbf541ff (0x7f38 bytes) 2024-09-12T09:30:51.685 INFO:teuthology.orchestra.run.smithi007.stdout:2492 writesame 0xe44e56f thru 0xe455d73 data_size 0x5(0x7805 bytes) 2024-09-12T09:30:51.685 INFO:teuthology.orchestra.run.smithi007.stdout:2493 write 0x4106242 thru 0x41137ad (0xd56c bytes) 2024-09-12T09:30:51.689 INFO:teuthology.orchestra.run.smithi007.stdout:2496 read 0x2a8ff51 thru 0x2a93ee0 (0x3f90 bytes) 2024-09-12T09:30:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:2498 punch from 0x6e007d0 to 0x6e0c988, (0xc1b8 bytes) 2024-09-12T09:30:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:2501 writesame 0x1b9a442 thru 0x1b9f32f data_size 0x1(0x4eee bytes) 2024-09-12T09:30:51.707 INFO:teuthology.orchestra.run.smithi007.stdout:2502 write 0xa010a94 thru 0xa016663 (0x5bd0 bytes) 2024-09-12T09:30:51.711 INFO:teuthology.orchestra.run.smithi007.stdout:2503 read 0x5fa2107 thru 0x5fb204e (0xff48 bytes) 2024-09-12T09:30:51.712 INFO:teuthology.orchestra.run.smithi007.stdout:2504 write 0x1123c13 thru 0x112c17f (0x856d bytes) 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2478 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14180352~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.315291576000007, 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049561444000000003, 2024-09-12T09:30:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2478 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397607+0000", 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397608+0000", 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397613+0000", 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397634+0000", 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0466999999999999e-05 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397647+0000", 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3597e-05 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446751+0000", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049103945000000003 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.446951+0000", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000199199 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447168+0000", 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021793600000000001 2024-09-12T09:30:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.315232166000001, 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.315175580000002, 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.315107267000002, 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.315049080999998, 2024-09-12T09:30:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314988548000002, 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314928547999997, 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314872199000007, 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314812888999995, 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:51.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314757255999993, 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:51.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314700873000007, 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314644302999994, 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:51.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314587887000002, 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:51.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314532393999997, 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314444381000001, 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:51.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:51.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.314379938000002, 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:51.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.467932152000003, 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:51.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:51.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:51.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.284515966999997, 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:51.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.168449864000003, 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:51.803 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:51.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:51.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.055420005999999, 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:51.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:51.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:51.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:51.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:51.816 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:51.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:2505 writesame 0x8546119 thru 0x854d003 data_size 0xf(0x6eeb bytes) 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:2506 writesame 0x212fa86 thru 0x213b2be data_size 0x1(0xb839 bytes) 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:2508 writesame 0x5e1df77 thru 0x5e21e4e data_size 0x4(0x3ed8 bytes) 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:2510 writesame 0x439942 thru 0x4482c5 data_size 0xa(0xe984 bytes) 2024-09-12T09:30:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:2511 read 0x86b42aa thru 0x86bd63c (0x9393 bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2514 read 0x7cd535b thru 0x7cd6ffa (0x1ca0 bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2515 write 0x21ad795 thru 0x21b30c3 (0x592f bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2516 punch from 0xb15fc4a to 0xb167d02, (0x80b8 bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2517 read 0x1691381 thru 0x1693f9d (0x2c1d bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2518 write 0x8cd4b7 thru 0x8d8287 (0xadd1 bytes) 2024-09-12T09:30:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:2522 read 0x925e4c5 thru 0x926bab6 (0xd5f2 bytes) 2024-09-12T09:30:51.820 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:51.816+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:51.821 INFO:teuthology.orchestra.run.smithi007.stdout:2525 trunc from 0xe455d74 to 0x568426b 2024-09-12T09:30:51.844 INFO:teuthology.orchestra.run.smithi007.stdout:2529 read 0x319fb4c thru 0x31ab445 (0xb8fa bytes) 2024-09-12T09:30:51.845 INFO:teuthology.orchestra.run.smithi007.stdout:2530 punch from 0x49f3022 to 0x49f4873, (0x1851 bytes) 2024-09-12T09:30:51.846 INFO:teuthology.orchestra.run.smithi007.stdout:2532 punch from 0x425d870 to 0x426637d, (0x8b0d bytes) 2024-09-12T09:30:51.846 INFO:teuthology.orchestra.run.smithi007.stdout:2533 punch from 0x24e782b to 0x24f5b19, (0xe2ee bytes) 2024-09-12T09:30:51.855 INFO:teuthology.orchestra.run.smithi007.stdout:2536 trunc from 0x568426b to 0x245eb9 2024-09-12T09:30:51.894 INFO:teuthology.orchestra.run.smithi007.stdout:2538 trunc from 0x245eb9 to 0xa96b4ab 2024-09-12T09:30:51.903 INFO:teuthology.orchestra.run.smithi007.stdout:2539 read 0x401c04f thru 0x4028bd0 (0xcb82 bytes) 2024-09-12T09:30:51.912 INFO:teuthology.orchestra.run.smithi007.stdout:2540 writesame 0xab7fa5c thru 0xab8cc8f data_size 0x16(0xd234 bytes) 2024-09-12T09:30:51.912 INFO:teuthology.orchestra.run.smithi007.stdout:2541 read 0x56d06f1 thru 0x56d443c (0x3d4c bytes) 2024-09-12T09:30:51.918 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:51.916+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:51.926 INFO:teuthology.orchestra.run.smithi007.stdout:2543 write 0xd6d393f thru 0xd6d7457 (0x3b19 bytes) 2024-09-12T09:30:51.933 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.933 INFO:teuthology.orchestra.run.smithi007.stdout:2545 read 0x8f83088 thru 0x8f8c009 (0x8f82 bytes) 2024-09-12T09:30:51.938 INFO:teuthology.orchestra.run.smithi007.stdout:2546 trunc from 0xd6d7458 to 0xb6a180c 2024-09-12T09:30:51.962 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:51.962 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:51.969 INFO:teuthology.orchestra.run.smithi007.stdout:2548 trunc from 0xb6a180c to 0x764d8d9 2024-09-12T09:30:51.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:51.982 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:51.982 INFO:teuthology.orchestra.run.smithi007.stdout:2550 clone 28 order 18 su 131072 sc 15 2024-09-12T09:30:52.020 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:52.019+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:52.046 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T09:30:52.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T09:30:52.120 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:52.120 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:52.120 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:52.120 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:52.120 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:52.121 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:52.120+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:52.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:52.222 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:52.219+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:52.227 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone27 intact 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.175268268000004, 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:30:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:30:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:30:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073932726000002, 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:30:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073907274, 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:52.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:30:52.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073882421999997, 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:30:52.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073857083999997, 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:30:52.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073826054000001, 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:30:52.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073800892999998, 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:30:52.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.352 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073774993000001, 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:52.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073751387000002, 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:30:52.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:30:52.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073726874999998, 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:30:52.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:52.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073702437000001, 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:52.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:30:52.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073677850000003, 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:30:52.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073653833999998, 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:52.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073625991999997, 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:30:52.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:30:52.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073601146999998, 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:30:52.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:30:52.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:52.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073576420999999, 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:52.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:30:52.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.073551924, 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:30:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:30:52.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.067746053, 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:30:52.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:30:52.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.922794089, 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:30:52.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:30:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:30:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:52.410 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.105297290999999, 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:52.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:52.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.416 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:52.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:52.418 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:52.416+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:52.518 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:52.516+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:52.531 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-12T09:30:52.569 INFO:teuthology.orchestra.run.smithi007.stdout:2551 writesame 0xbd4028b thru 0xbd434df data_size 0x5(0x3255 bytes) 2024-09-12T09:30:52.569 INFO:teuthology.orchestra.run.smithi007.stdout:2552 read 0x46deca3 thru 0x46e9099 (0xa3f7 bytes) 2024-09-12T09:30:52.569 INFO:teuthology.orchestra.run.smithi007.stdout:2553 write 0x196a685 thru 0x1971cbb (0x7637 bytes) 2024-09-12T09:30:52.577 INFO:teuthology.orchestra.run.smithi007.stdout:2554 punch from 0x1318658 to 0x1327048, (0xe9f0 bytes) 2024-09-12T09:30:52.577 INFO:teuthology.orchestra.run.smithi007.stdout:2555 write 0x7dd6e7c thru 0x7dd7899 (0xa1e bytes) 2024-09-12T09:30:52.582 INFO:teuthology.orchestra.run.smithi007.stdout:2556 read 0x96e4f69 thru 0x96e9921 (0x49b9 bytes) 2024-09-12T09:30:52.583 INFO:teuthology.orchestra.run.smithi007.stdout:2557 read 0xadb9862 thru 0xadc4d11 (0xb4b0 bytes) 2024-09-12T09:30:52.588 INFO:teuthology.orchestra.run.smithi007.stdout:2559 trunc from 0xbd434e0 to 0x81603f0 2024-09-12T09:30:52.589 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:52.590 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:52.603 INFO:teuthology.orchestra.run.smithi007.stdout:2560 punch from 0x8b5827 to 0x8b8883, (0x305c bytes) 2024-09-12T09:30:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:2561 read 0x22a6a77 thru 0x22a7c49 (0x11d3 bytes) 2024-09-12T09:30:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:2562 punch from 0x3b2878a to 0x3b2da96, (0x530c bytes) 2024-09-12T09:30:52.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:52.628 INFO:teuthology.orchestra.run.smithi007.stdout:2563 write 0xe79b97a thru 0xe7a6fcb (0xb652 bytes) 2024-09-12T09:30:52.628 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:52.628+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:2564 punch from 0xa84d722 to 0xa853c31, (0x650f bytes) 2024-09-12T09:30:52.636 INFO:teuthology.orchestra.run.smithi007.stdout:2565 writesame 0xb6c233d thru 0xb6c84aa data_size 0x3(0x616e bytes) 2024-09-12T09:30:52.645 INFO:teuthology.orchestra.run.smithi007.stdout:2566 writesame 0x9546c8f thru 0x955253d data_size 0x1(0xb8af bytes) 2024-09-12T09:30:52.703 INFO:teuthology.orchestra.run.smithi007.stdout:2568 read 0x2942585 thru 0x294ee4f (0xc8cb bytes) 2024-09-12T09:30:52.704 INFO:teuthology.orchestra.run.smithi007.stdout:2571 writesame 0x484f00d thru 0x4858e89 data_size 0x1(0x9e7d bytes) 2024-09-12T09:30:52.719 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:52.716+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:52.758 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:52.758 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:52.761 INFO:teuthology.orchestra.run.smithi007.stdout:2574 punch from 0xc0c9a1d to 0xc0cda8f, (0x4072 bytes) 2024-09-12T09:30:52.761 INFO:teuthology.orchestra.run.smithi007.stdout:2575 write 0x2527b24 thru 0x25296f9 (0x1bd6 bytes) 2024-09-12T09:30:52.771 INFO:teuthology.orchestra.run.smithi007.stdout:2576 punch from 0x1a4eac to 0x1ae4fe, (0x9652 bytes) 2024-09-12T09:30:52.771 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:52.772 INFO:teuthology.orchestra.run.smithi007.stdout:2578 punch from 0xc1408e2 to 0xc1456a6, (0x4dc4 bytes) 2024-09-12T09:30:52.772 INFO:teuthology.orchestra.run.smithi007.stdout:2579 write 0x59fbdd4 thru 0x5a053f2 (0x961f bytes) 2024-09-12T09:30:52.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:52.781 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:52.781 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:52.781 INFO:teuthology.orchestra.run.smithi007.stdout:2583 write 0x2418b3b thru 0x241e194 (0x565a bytes) 2024-09-12T09:30:52.811 INFO:teuthology.orchestra.run.smithi007.stdout:2585 trunc from 0xe7a6fcc to 0x104bb20 2024-09-12T09:30:52.820 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:52.819+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:52.892 INFO:teuthology.orchestra.run.smithi007.stdout:2586 write 0xd2402b1 thru 0xd246d0c (0x6a5c bytes) 2024-09-12T09:30:52.900 INFO:teuthology.orchestra.run.smithi007.stdout:2587 read 0x6ebe88b thru 0x6ec0e6e (0x25e4 bytes) 2024-09-12T09:30:52.900 INFO:teuthology.orchestra.run.smithi007.stdout:2588 punch from 0x65ef3aa to 0x65f84cf, (0x9125 bytes) 2024-09-12T09:30:52.900 INFO:teuthology.orchestra.run.smithi007.stdout:2589 writesame 0x8debe83 thru 0x8df8bb1 data_size 0x1(0xcd2f bytes) 2024-09-12T09:30:52.921 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:52.920+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.722938628000001, 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:30:52.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:30:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.707967899, 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:30:52.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:52.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:30:52.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.391120895, 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:30:52.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:52.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.391074536000001, 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:30:52.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:30:52.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390967316000001, 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:52.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:30:52.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390873825, 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:30:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:30:52.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390807312, 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:30:52.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390746502999999, 2024-09-12T09:30:52.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:30:52.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:30:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390687210999999, 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:52.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:30:52.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390626701999999, 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.977 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390548646999999, 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:30:52.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:30:52.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390463272000002, 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:30:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390394953000001, 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:30:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390302368, 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:30:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390246459, 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:30:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:30:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390214136000001, 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:53.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:53.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:30:53.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390180543, 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:53.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:30:53.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390129601000002, 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:53.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:30:53.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.390031445999998, 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:53.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:53.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:53.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.389974351999999, 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:53.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:53.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:30:53.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:53.025 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:53.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:2591 punch from 0x3a451ea to 0x3a4a70f, (0x5525 bytes) 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:2594 write 0x2e88d67 thru 0x2e8e80d (0x5aa7 bytes) 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:2595 writesame 0x663a6a6 thru 0x664365c data_size 0x1(0x8fb7 bytes) 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:2596 writesame 0x99218ba thru 0x992e4f9 data_size 0x98(0xcc40 bytes) 2024-09-12T09:30:53.028 INFO:teuthology.orchestra.run.smithi007.stdout:2597 write 0xe172009 thru 0xe176b03 (0x4afb bytes) 2024-09-12T09:30:53.029 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:53.023+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:53.029 INFO:teuthology.orchestra.run.smithi007.stdout:2598 writesame 0xb0a87c4 thru 0xb0ab252 data_size 0x1(0x2a8f bytes) 2024-09-12T09:30:53.048 INFO:teuthology.orchestra.run.smithi007.stdout:2599 read 0x57c315c thru 0x57c9435 (0x62da bytes) 2024-09-12T09:30:53.062 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T09:30:53.069 INFO:teuthology.orchestra.run.smithi007.stdout:2600 write 0xe26dcde thru 0xe27b30b (0xd62e bytes) 2024-09-12T09:30:53.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:30:53.075 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:53.075 INFO:teuthology.orchestra.run.smithi007.stdout:2603 write 0x207380c thru 0x207a1f7 (0x69ec bytes) 2024-09-12T09:30:53.085 INFO:teuthology.orchestra.run.smithi007.stdout:2604 read 0x24fde11 thru 0x250c4ea (0xe6da bytes) 2024-09-12T09:30:53.085 INFO:teuthology.orchestra.run.smithi007.stdout:2605 writesame 0xc6812df thru 0xc686db7 data_size 0x1(0x5ad9 bytes) 2024-09-12T09:30:53.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T09:30:53.106 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:53.106 INFO:teuthology.orchestra.run.smithi007.stdout:2607 write 0x959fd08 thru 0x95a36ab (0x39a4 bytes) 2024-09-12T09:30:53.113 INFO:teuthology.orchestra.run.smithi007.stdout:2608 punch from 0x415c3d9 to 0x4164ab1, (0x86d8 bytes) 2024-09-12T09:30:53.113 INFO:teuthology.orchestra.run.smithi007.stdout:2609 punch from 0x65901c6 to 0x6590d1b, (0xb55 bytes) 2024-09-12T09:30:53.113 INFO:teuthology.orchestra.run.smithi007.stdout:2610 punch from 0x94714d0 to 0x947356a, (0x209a bytes) 2024-09-12T09:30:53.113 INFO:teuthology.orchestra.run.smithi007.stdout:2613 write 0x8b126d4 thru 0x8b190df (0x6a0c bytes) 2024-09-12T09:30:53.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:53.119 INFO:teuthology.orchestra.run.smithi007.stdout:2614 punch from 0xb97ad68 to 0xb989324, (0xe5bc bytes) 2024-09-12T09:30:53.119 INFO:teuthology.orchestra.run.smithi007.stdout:2616 punch from 0x82fd773 to 0x8303de7, (0x6674 bytes) 2024-09-12T09:30:53.119 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:53.119 INFO:teuthology.orchestra.run.smithi007.stdout:2618 write 0x59c4c28 thru 0x59d2dfb (0xe1d4 bytes) 2024-09-12T09:30:53.126 INFO:teuthology.orchestra.run.smithi007.stdout:2619 read 0x50cf3f6 thru 0x50dd549 (0xe154 bytes) 2024-09-12T09:30:53.127 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:53.124+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:53.136 INFO:teuthology.orchestra.run.smithi007.stdout:2620 trunc from 0xe27b30c to 0x857233b 2024-09-12T09:30:53.163 INFO:teuthology.orchestra.run.smithi007.stdout:2621 write 0x4a134a2 thru 0x4a189bb (0x551a bytes) 2024-09-12T09:30:53.172 INFO:teuthology.orchestra.run.smithi007.stdout:2622 trunc from 0x857233b to 0x71863f5 2024-09-12T09:30:53.180 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:53.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:53.181 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:53.181 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:53.183 INFO:teuthology.orchestra.run.smithi007.stdout:2623 write 0x583e860 thru 0x5846271 (0x7a12 bytes) 2024-09-12T09:30:53.201 INFO:teuthology.orchestra.run.smithi007.stdout:2624 trunc from 0x71863f5 to 0xd24dacb 2024-09-12T09:30:53.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:53.212 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:53.212 INFO:teuthology.orchestra.run.smithi007.stdout:2627 read 0x7b7534 thru 0x7bce28 (0x58f5 bytes) 2024-09-12T09:30:53.214 INFO:teuthology.orchestra.run.smithi007.stdout:2629 trunc from 0xd24dacb to 0xda6e08c 2024-09-12T09:30:53.227 INFO:teuthology.orchestra.run.smithi007.stdout:2631 writesame 0x6ff8f33 thru 0x6fffceb data_size 0x3(0x6db9 bytes) 2024-09-12T09:30:53.227 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:53.223+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:53.242 INFO:teuthology.orchestra.run.smithi007.stdout:2633 writesame 0x132f8f3 thru 0x13397a8 data_size 0x55(0x9eb6 bytes) 2024-09-12T09:30:53.248 INFO:teuthology.orchestra.run.smithi007.stdout:2637 write 0x511223f thru 0x511e8b5 (0xc677 bytes) 2024-09-12T09:30:53.254 INFO:teuthology.orchestra.run.smithi007.stdout:2638 punch from 0xb32cb14 to 0xb32fae9, (0x2fd5 bytes) 2024-09-12T09:30:53.274 INFO:teuthology.orchestra.run.smithi007.stdout:2639 trunc from 0xda6e08c to 0x1c1bb7c 2024-09-12T09:30:53.306 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:53.328 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:53.327+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:53.332 INFO:teuthology.orchestra.run.smithi007.stdout:2641 write 0xe30d79e thru 0xe313ec7 (0x672a bytes) 2024-09-12T09:30:53.332 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:53.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:53.347 INFO:teuthology.orchestra.run.smithi007.stdout:2644 trunc from 0xe313ec8 to 0x82d8c85 2024-09-12T09:30:53.367 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:53.368 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:53.368 INFO:teuthology.orchestra.run.smithi007.stdout:2645 read 0x7a06674 thru 0x7a07bb1 (0x153e bytes) 2024-09-12T09:30:53.375 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:53.375 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:53.376 INFO:teuthology.orchestra.run.smithi007.stdout:2646 trunc from 0x82d8c85 to 0x290ca84 2024-09-12T09:30:53.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:53.401 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:53.401 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:53.407 INFO:teuthology.orchestra.run.smithi007.stdout:2649 write 0x6f671c8 thru 0x6f6fd05 (0x8b3e bytes) 2024-09-12T09:30:53.413 INFO:teuthology.orchestra.run.smithi007.stdout:2650 punch from 0x6d13ef6 to 0x6d1b1f2, (0x72fc bytes) 2024-09-12T09:30:53.419 INFO:teuthology.orchestra.run.smithi007.stdout:2651 trunc from 0x6f6fd06 to 0x2fb0821 2024-09-12T09:30:53.424 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:53.429 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:53.428+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:53.439 INFO:teuthology.orchestra.run.smithi007.stdout:2653 read 0x2ccaa8f thru 0x2cd7105 (0xc677 bytes) 2024-09-12T09:30:53.446 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:53.468 INFO:teuthology.orchestra.run.smithi007.stdout:2654 write 0xdb960a8 thru 0xdb9ec10 (0x8b69 bytes) 2024-09-12T09:30:53.468 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:2655 punch from 0x6fdc286 to 0x6fea54f, (0xe2c9 bytes) 2024-09-12T09:30:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:2656 read 0x9a950d1 thru 0x9aa446a (0xf39a bytes) 2024-09-12T09:30:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:2657 read 0x36345e5 thru 0x36437f6 (0xf212 bytes) 2024-09-12T09:30:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:2658 write 0x14c9d7 thru 0x15a4fd (0xdb27 bytes) 2024-09-12T09:30:53.481 INFO:teuthology.orchestra.run.smithi007.stdout:2659 read 0x5ba0384 thru 0x5ba32bf (0x2f3c bytes) 2024-09-12T09:30:53.491 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:53.491 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:53.493 INFO:teuthology.orchestra.run.smithi007.stdout:2660 write 0xe84a2ed thru 0xe85498f (0xa6a3 bytes) 2024-09-12T09:30:53.499 INFO:teuthology.orchestra.run.smithi007.stdout:2661 read 0xc9b8665 thru 0xc9c28f8 (0xa294 bytes) 2024-09-12T09:30:53.499 INFO:teuthology.orchestra.run.smithi007.stdout:2663 writesame 0x18a5c20 thru 0x18aaf46 data_size 0x1(0x5327 bytes) 2024-09-12T09:30:53.528 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:53.529 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:53.529 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:53.528+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:53.531 INFO:teuthology.orchestra.run.smithi007.stdout:2664 read 0x7ecdd2f thru 0x7ecf426 (0x16f8 bytes) 2024-09-12T09:30:53.531 INFO:teuthology.orchestra.run.smithi007.stdout:2665 clone 29 order 18 su 16384 sc 6 2024-09-12T09:30:53.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:53.631 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:53.627+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:53.709 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:53.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:53.768 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:53.768+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:53.833 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:53.832+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:53.847 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:53.847 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:53.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:53.934 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:53.931+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:54.028 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.028 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.035 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:54.031+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:54.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:54.064 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:30:54.064 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T09:30:54.090 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T09:30:54.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:30:54.136 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:54.135+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:54.180 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.180 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:54.234 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone28 2024-09-12T09:30:54.237 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:54.236+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:54.338 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:54.336+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:54.366 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.366 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:54.439 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:54.436+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:54.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:54.455 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:30:54.473 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-12T09:30:54.065579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1279394862,"num_objects":1203,"num_object_clones":285,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":476,"num_read":165430,"num_read_kb":1194026,"num_write":8924,"num_write_kb":696228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":347624051,"num_keys_recovered":207,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6876,"ondisk_log_size":6876,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":280260,"kb_used_data":72076,"kb_used_omap":9,"kb_used_meta":207798,"kb_avail":565950780,"statfs":{"total":579820584960,"available":579533598720,"internally_reserved":0,"allocated":73805824,"data_stored":975673554,"data_compressed":8890739,"data_compressed_allocated":60362752,"data_compressed_original":967925608,"omap_allocated":9563,"internal_metadata":212785829},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62019057,"num_objects":74,"num_object_clones":35,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":11,"num_read":4366,"num_read_kb":46492,"num_write":767,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071719"},"pg_stats":[{"pgid":"3.1f","version":"86'417","reported_seq":4869,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049896+0000","last_change":"2024-09-12T09:29:58.167241+0000","last_active":"2024-09-12T09:30:51.049896+0000","last_peered":"2024-09-12T09:30:51.049896+0000","last_clean":"2024-09-12T09:30:51.049896+0000","last_became_active":"2024-09-12T09:29:58.165903+0000","last_became_peered":"2024-09-12T09:29:58.165903+0000","last_unstale":"2024-09-12T09:30:51.049896+0000","last_undegraded":"2024-09-12T09:30:51.049896+0000","last_fullsized":"2024-09-12T09:30:51.049896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:31.153821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":86565125,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":5570,"num_read_kb":88595,"num_write":585,"num_write_kb":45291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":4329673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"87'403","reported_seq":4096,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052342+0000","last_change":"2024-09-12T09:30:01.194793+0000","last_active":"2024-09-12T09:30:51.052342+0000","last_peered":"2024-09-12T09:30:51.052342+0000","last_clean":"2024-09-12T09:30:51.052342+0000","last_became_active":"2024-09-12T09:30:01.169430+0000","last_became_peered":"2024-09-12T09:30:01.169430+0000","last_unstale":"2024-09-12T09:30:51.052342+0000","last_undegraded":"2024-09-12T09:30:51.052342+0000","last_fullsized":"2024-09-12T09:30:51.052342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:46.408572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":41789349,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":5693,"num_read_kb":33945,"num_write":558,"num_write_kb":33688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":352930,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"82'162","reported_seq":3649,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055729+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:30:51.055729+0000","last_peered":"2024-09-12T09:30:51.055729+0000","last_clean":"2024-09-12T09:30:51.055729+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:30:51.055729+0000","last_undegraded":"2024-09-12T09:30:51.055729+0000","last_fullsized":"2024-09-12T09:30:51.055729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:02.526323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":43645672,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":3818,"num_read_kb":65645,"num_write":144,"num_write_kb":35929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"87'179","reported_seq":3874,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055407+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:30:51.055407+0000","last_peered":"2024-09-12T09:30:51.055407+0000","last_clean":"2024-09-12T09:30:51.055407+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:30:51.055407+0000","last_undegraded":"2024-09-12T09:30:51.055407+0000","last_fullsized":"2024-09-12T09:30:51.055407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:32.316709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":19972439,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4199,"num_read_kb":33781,"num_write":196,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'169","reported_seq":4815,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049540+0000","last_change":"2024-09-12T09:29:55.231536+0000","last_active":"2024-09-12T09:30:51.049540+0000","last_peered":"2024-09-12T09:30:51.049540+0000","last_clean":"2024-09-12T09:30:51.049540+0000","last_became_active":"2024-09-12T09:29:55.137238+0000","last_became_peered":"2024-09-12T09:29:55.137238+0000","last_unstale":"2024-09-12T09:30:51.049540+0000","last_undegraded":"2024-09-12T09:30:51.049540+0000","last_fullsized":"2024-09-12T09:30:51.049540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":27142706,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":5284,"num_read_kb":12908,"num_write":179,"num_write_kb":4190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1206936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"87'200","reported_seq":2321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053098+0000","last_change":"2024-09-12T09:30:46.979355+0000","last_active":"2024-09-12T09:30:52.053098+0000","last_peered":"2024-09-12T09:30:52.053098+0000","last_clean":"2024-09-12T09:30:52.053098+0000","last_became_active":"2024-09-12T09:30:46.978472+0000","last_became_peered":"2024-09-12T09:30:46.978472+0000","last_unstale":"2024-09-12T09:30:52.053098+0000","last_undegraded":"2024-09-12T09:30:52.053098+0000","last_fullsized":"2024-09-12T09:30:52.053098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:44.155070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":9197978,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2365,"num_read_kb":9518,"num_write":204,"num_write_kb":30034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2195003,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'108","reported_seq":711,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052000+0000","last_change":"2024-09-12T09:30:44.080862+0000","last_active":"2024-09-12T09:30:51.052000+0000","last_peered":"2024-09-12T09:30:51.052000+0000","last_clean":"2024-09-12T09:30:51.052000+0000","last_became_active":"2024-09-12T09:30:43.967238+0000","last_became_peered":"2024-09-12T09:30:43.967238+0000","last_unstale":"2024-09-12T09:30:51.052000+0000","last_undegraded":"2024-09-12T09:30:51.052000+0000","last_fullsized":"2024-09-12T09:30:51.052000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:29.283242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":24759249,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":641,"num_read_kb":4170,"num_write":92,"num_write_kb":1812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":970649,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"87'141","reported_seq":1291,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.051983+0000","last_change":"2024-09-12T09:29:55.177304+0000","last_active":"2024-09-12T09:30:51.051983+0000","last_peered":"2024-09-12T09:30:51.051983+0000","last_clean":"2024-09-12T09:30:51.051983+0000","last_became_active":"2024-09-12T09:29:55.134985+0000","last_became_peered":"2024-09-12T09:29:55.134985+0000","last_unstale":"2024-09-12T09:30:51.051983+0000","last_undegraded":"2024-09-12T09:30:51.051983+0000","last_fullsized":"2024-09-12T09:30:51.051983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:23.074794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":11573008,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1170,"num_read_kb":5400,"num_write":151,"num_write_kb":4735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":524806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"87'130","reported_seq":2461,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049763+0000","last_change":"2024-09-12T09:29:55.167682+0000","last_active":"2024-09-12T09:30:51.049763+0000","last_peered":"2024-09-12T09:30:51.049763+0000","last_clean":"2024-09-12T09:30:51.049763+0000","last_became_active":"2024-09-12T09:29:55.136789+0000","last_became_peered":"2024-09-12T09:29:55.136789+0000","last_unstale":"2024-09-12T09:30:51.049763+0000","last_undegraded":"2024-09-12T09:30:51.049763+0000","last_fullsized":"2024-09-12T09:30:51.049763+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:10:27.604010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":42465040,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":2648,"num_read_kb":47032,"num_write":229,"num_write_kb":36221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8413,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"88'226","reported_seq":13691,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055728+0000","last_change":"2024-09-12T09:30:33.669800+0000","last_active":"2024-09-12T09:30:51.055728+0000","last_peered":"2024-09-12T09:30:51.055728+0000","last_clean":"2024-09-12T09:30:51.055728+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:30:51.055728+0000","last_undegraded":"2024-09-12T09:30:51.055728+0000","last_fullsized":"2024-09-12T09:30:51.055728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:00.826841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":63441202,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":13510,"num_read_kb":69961,"num_write":304,"num_write_kb":27133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'482","reported_seq":5563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064643+0000","last_change":"2024-09-12T09:30:34.672921+0000","last_active":"2024-09-12T09:30:52.064643+0000","last_peered":"2024-09-12T09:30:52.064643+0000","last_clean":"2024-09-12T09:30:52.064643+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:30:52.064643+0000","last_undegraded":"2024-09-12T09:30:52.064643+0000","last_fullsized":"2024-09-12T09:30:52.064643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:38.199970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":83238189,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":6392,"num_read_kb":77369,"num_write":738,"num_write_kb":42344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"89'524","reported_seq":8201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053674+0000","last_change":"2024-09-12T09:30:33.668610+0000","last_active":"2024-09-12T09:30:52.053674+0000","last_peered":"2024-09-12T09:30:52.053674+0000","last_clean":"2024-09-12T09:30:52.053674+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:30:52.053674+0000","last_undegraded":"2024-09-12T09:30:52.053674+0000","last_fullsized":"2024-09-12T09:30:52.053674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:00.155518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":22543438,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":9525,"num_read_kb":20252,"num_write":863,"num_write_kb":23530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064044+0000","last_change":"2024-09-12T09:30:30.871108+0000","last_active":"2024-09-12T09:30:52.064044+0000","last_peered":"2024-09-12T09:30:52.064044+0000","last_clean":"2024-09-12T09:30:52.064044+0000","last_became_active":"2024-09-12T09:30:30.870346+0000","last_became_peered":"2024-09-12T09:30:30.870346+0000","last_unstale":"2024-09-12T09:30:52.064044+0000","last_undegraded":"2024-09-12T09:30:52.064044+0000","last_fullsized":"2024-09-12T09:30:52.064044+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:23.248076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"85'122","reported_seq":4862,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049410+0000","last_change":"2024-09-12T09:30:46.982993+0000","last_active":"2024-09-12T09:30:51.049410+0000","last_peered":"2024-09-12T09:30:51.049410+0000","last_clean":"2024-09-12T09:30:51.049410+0000","last_became_active":"2024-09-12T09:30:46.981203+0000","last_became_peered":"2024-09-12T09:30:46.981203+0000","last_unstale":"2024-09-12T09:30:51.049410+0000","last_undegraded":"2024-09-12T09:30:51.049410+0000","last_fullsized":"2024-09-12T09:30:51.049410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:06.188499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":21006461,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":5201,"num_read_kb":19833,"num_write":107,"num_write_kb":4869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":4829848,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053870+0000","last_change":"2024-09-12T09:30:43.963886+0000","last_active":"2024-09-12T09:30:52.053870+0000","last_peered":"2024-09-12T09:30:52.053870+0000","last_clean":"2024-09-12T09:30:52.053870+0000","last_became_active":"2024-09-12T09:30:43.962849+0000","last_became_peered":"2024-09-12T09:30:43.962849+0000","last_unstale":"2024-09-12T09:30:52.053870+0000","last_undegraded":"2024-09-12T09:30:52.053870+0000","last_fullsized":"2024-09-12T09:30:52.053870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:38.942689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"85'138","reported_seq":2041,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063785+0000","last_change":"2024-09-12T09:30:33.673235+0000","last_active":"2024-09-12T09:30:52.063785+0000","last_peered":"2024-09-12T09:30:52.063785+0000","last_clean":"2024-09-12T09:30:52.063785+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:30:52.063785+0000","last_undegraded":"2024-09-12T09:30:52.063785+0000","last_fullsized":"2024-09-12T09:30:52.063785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:12:08.342915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":25079818,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":1788,"num_read_kb":14526,"num_write":197,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064285+0000","last_change":"2024-09-12T09:30:30.875450+0000","last_active":"2024-09-12T09:30:52.064285+0000","last_peered":"2024-09-12T09:30:52.064285+0000","last_clean":"2024-09-12T09:30:52.064285+0000","last_became_active":"2024-09-12T09:30:30.874655+0000","last_became_peered":"2024-09-12T09:30:30.874655+0000","last_unstale":"2024-09-12T09:30:52.064285+0000","last_undegraded":"2024-09-12T09:30:52.064285+0000","last_fullsized":"2024-09-12T09:30:52.064285+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:30.720674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"87'140","reported_seq":4852,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049566+0000","last_change":"2024-09-12T09:30:46.985105+0000","last_active":"2024-09-12T09:30:51.049566+0000","last_peered":"2024-09-12T09:30:51.049566+0000","last_clean":"2024-09-12T09:30:51.049566+0000","last_became_active":"2024-09-12T09:30:46.983676+0000","last_became_peered":"2024-09-12T09:30:46.983676+0000","last_unstale":"2024-09-12T09:30:51.049566+0000","last_undegraded":"2024-09-12T09:30:51.049566+0000","last_fullsized":"2024-09-12T09:30:51.049566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:18.469021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":24332273,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":4487,"num_read_kb":20388,"num_write":222,"num_write_kb":43382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":7432750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049288+0000","last_change":"2024-09-12T09:30:43.961453+0000","last_active":"2024-09-12T09:30:51.049288+0000","last_peered":"2024-09-12T09:30:51.049288+0000","last_clean":"2024-09-12T09:30:51.049288+0000","last_became_active":"2024-09-12T09:30:43.960994+0000","last_became_peered":"2024-09-12T09:30:43.960994+0000","last_unstale":"2024-09-12T09:30:51.049288+0000","last_undegraded":"2024-09-12T09:30:51.049288+0000","last_fullsized":"2024-09-12T09:30:51.049288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.668460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"87'133","reported_seq":13344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055958+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:30:51.055958+0000","last_peered":"2024-09-12T09:30:51.055958+0000","last_clean":"2024-09-12T09:30:51.055958+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:30:51.055958+0000","last_undegraded":"2024-09-12T09:30:51.055958+0000","last_fullsized":"2024-09-12T09:30:51.055958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:50.066462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":35639524,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":13338,"num_read_kb":55483,"num_write":116,"num_write_kb":11232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"88'323","reported_seq":4256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053805+0000","last_change":"2024-09-12T09:29:55.204077+0000","last_active":"2024-09-12T09:30:52.053805+0000","last_peered":"2024-09-12T09:30:52.053805+0000","last_clean":"2024-09-12T09:30:52.053805+0000","last_became_active":"2024-09-12T09:29:55.135753+0000","last_became_peered":"2024-09-12T09:29:55.135753+0000","last_unstale":"2024-09-12T09:30:52.053805+0000","last_undegraded":"2024-09-12T09:30:52.053805+0000","last_fullsized":"2024-09-12T09:30:52.053805+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:52.900351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":25327632,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":5188,"num_read_kb":19677,"num_write":397,"num_write_kb":9398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":57888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":165,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055184+0000","last_change":"2024-09-12T09:30:43.969129+0000","last_active":"2024-09-12T09:30:51.055184+0000","last_peered":"2024-09-12T09:30:51.055184+0000","last_clean":"2024-09-12T09:30:51.055184+0000","last_became_active":"2024-09-12T09:30:43.962175+0000","last_became_peered":"2024-09-12T09:30:43.962175+0000","last_unstale":"2024-09-12T09:30:51.055184+0000","last_undegraded":"2024-09-12T09:30:51.055184+0000","last_fullsized":"2024-09-12T09:30:51.055184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:00:15.922849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"87'169","reported_seq":3503,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.048932+0000","last_change":"2024-09-12T09:30:46.984392+0000","last_active":"2024-09-12T09:30:51.048932+0000","last_peered":"2024-09-12T09:30:51.048932+0000","last_clean":"2024-09-12T09:30:51.048932+0000","last_became_active":"2024-09-12T09:30:46.982399+0000","last_became_peered":"2024-09-12T09:30:46.982399+0000","last_unstale":"2024-09-12T09:30:51.048932+0000","last_undegraded":"2024-09-12T09:30:51.048932+0000","last_fullsized":"2024-09-12T09:30:51.048932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:24.471861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":45358330,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":3125,"num_read_kb":34218,"num_write":142,"num_write_kb":14520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2593871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049222+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:30:51.049222+0000","last_peered":"2024-09-12T09:30:51.049222+0000","last_clean":"2024-09-12T09:30:51.049222+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:30:51.049222+0000","last_undegraded":"2024-09-12T09:30:51.049222+0000","last_fullsized":"2024-09-12T09:30:51.049222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'181","reported_seq":9732,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.048863+0000","last_change":"2024-09-12T09:29:55.163561+0000","last_active":"2024-09-12T09:30:51.048863+0000","last_peered":"2024-09-12T09:30:51.048863+0000","last_clean":"2024-09-12T09:30:51.048863+0000","last_became_active":"2024-09-12T09:29:55.134480+0000","last_became_peered":"2024-09-12T09:29:55.134480+0000","last_unstale":"2024-09-12T09:30:51.048863+0000","last_undegraded":"2024-09-12T09:30:51.048863+0000","last_fullsized":"2024-09-12T09:30:51.048863+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:05.743583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":38984553,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":10443,"num_read_kb":32779,"num_write":159,"num_write_kb":20082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2206098,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063507+0000","last_change":"2024-09-12T09:30:43.963299+0000","last_active":"2024-09-12T09:30:52.063507+0000","last_peered":"2024-09-12T09:30:52.063507+0000","last_clean":"2024-09-12T09:30:52.063507+0000","last_became_active":"2024-09-12T09:30:43.961820+0000","last_became_peered":"2024-09-12T09:30:43.961820+0000","last_unstale":"2024-09-12T09:30:52.063507+0000","last_undegraded":"2024-09-12T09:30:52.063507+0000","last_fullsized":"2024-09-12T09:30:52.063507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:20.604956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'122","reported_seq":4617,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055267+0000","last_change":"2024-09-12T09:30:12.711941+0000","last_active":"2024-09-12T09:30:51.055267+0000","last_peered":"2024-09-12T09:30:51.055267+0000","last_clean":"2024-09-12T09:30:51.055267+0000","last_became_active":"2024-09-12T09:30:12.711307+0000","last_became_peered":"2024-09-12T09:30:12.711307+0000","last_unstale":"2024-09-12T09:30:51.055267+0000","last_undegraded":"2024-09-12T09:30:51.055267+0000","last_fullsized":"2024-09-12T09:30:51.055267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:23.436691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":32879263,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":5336,"num_read_kb":41925,"num_write":139,"num_write_kb":10668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":154,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055294+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:30:51.055294+0000","last_peered":"2024-09-12T09:30:51.055294+0000","last_clean":"2024-09-12T09:30:51.055294+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:30:51.055294+0000","last_undegraded":"2024-09-12T09:30:51.055294+0000","last_fullsized":"2024-09-12T09:30:51.055294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063245+0000","last_change":"2024-09-12T09:30:30.889337+0000","last_active":"2024-09-12T09:30:52.063245+0000","last_peered":"2024-09-12T09:30:52.063245+0000","last_clean":"2024-09-12T09:30:52.063245+0000","last_became_active":"2024-09-12T09:30:30.873745+0000","last_became_peered":"2024-09-12T09:30:30.873745+0000","last_unstale":"2024-09-12T09:30:52.063245+0000","last_undegraded":"2024-09-12T09:30:52.063245+0000","last_fullsized":"2024-09-12T09:30:52.063245+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:57.848261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"85'232","reported_seq":3904,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.051982+0000","last_change":"2024-09-12T09:30:46.981798+0000","last_active":"2024-09-12T09:30:51.051982+0000","last_peered":"2024-09-12T09:30:51.051982+0000","last_clean":"2024-09-12T09:30:51.051982+0000","last_became_active":"2024-09-12T09:30:46.980108+0000","last_became_peered":"2024-09-12T09:30:46.980108+0000","last_unstale":"2024-09-12T09:30:51.051982+0000","last_undegraded":"2024-09-12T09:30:51.051982+0000","last_fullsized":"2024-09-12T09:30:51.051982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:43.658520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":38826208,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":5334,"num_read_kb":49163,"num_write":279,"num_write_kb":20701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":10746792,"num_keys_recovered":21,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"86'179","reported_seq":2078,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053350+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:30:52.053350+0000","last_peered":"2024-09-12T09:30:52.053350+0000","last_clean":"2024-09-12T09:30:52.053350+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:30:52.053350+0000","last_undegraded":"2024-09-12T09:30:52.053350+0000","last_fullsized":"2024-09-12T09:30:52.053350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:10.027724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":22133403,"num_objects":33,"num_object_clones":4,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":1974,"num_read_kb":11122,"num_write":193,"num_write_kb":8073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'178","reported_seq":4786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053015+0000","last_change":"2024-09-12T09:30:46.979031+0000","last_active":"2024-09-12T09:30:52.053015+0000","last_peered":"2024-09-12T09:30:52.053015+0000","last_clean":"2024-09-12T09:30:52.053015+0000","last_became_active":"2024-09-12T09:30:46.977847+0000","last_became_peered":"2024-09-12T09:30:46.977847+0000","last_unstale":"2024-09-12T09:30:52.053015+0000","last_undegraded":"2024-09-12T09:30:52.053015+0000","last_fullsized":"2024-09-12T09:30:52.053015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:10.148048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":28661649,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4680,"num_read_kb":11083,"num_write":254,"num_write_kb":5580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":7309669,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'268","reported_seq":5533,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052401+0000","last_change":"2024-09-12T09:29:55.239142+0000","last_active":"2024-09-12T09:30:51.052401+0000","last_peered":"2024-09-12T09:30:51.052401+0000","last_clean":"2024-09-12T09:30:51.052401+0000","last_became_active":"2024-09-12T09:29:55.135093+0000","last_became_peered":"2024-09-12T09:29:55.135093+0000","last_unstale":"2024-09-12T09:30:51.052401+0000","last_undegraded":"2024-09-12T09:30:51.052401+0000","last_fullsized":"2024-09-12T09:30:51.052401+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:40.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":73510413,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":5875,"num_read_kb":70738,"num_write":450,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1338871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"87'155","reported_seq":2253,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055665+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:30:51.055665+0000","last_peered":"2024-09-12T09:30:51.055665+0000","last_clean":"2024-09-12T09:30:51.055665+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:30:51.055665+0000","last_undegraded":"2024-09-12T09:30:51.055665+0000","last_fullsized":"2024-09-12T09:30:51.055665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:04.790845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":26022144,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3192,"num_read_kb":30963,"num_write":178,"num_write_kb":16002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"88'157","reported_seq":7209,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063799+0000","last_change":"2024-09-12T09:30:33.675916+0000","last_active":"2024-09-12T09:30:52.063799+0000","last_peered":"2024-09-12T09:30:52.063799+0000","last_clean":"2024-09-12T09:30:52.063799+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:30:52.063799+0000","last_undegraded":"2024-09-12T09:30:52.063799+0000","last_fullsized":"2024-09-12T09:30:52.063799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:32.494075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":23336760,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":7417,"num_read_kb":24518,"num_write":220,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"87'206","reported_seq":4687,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055465+0000","last_change":"2024-09-12T09:30:27.834395+0000","last_active":"2024-09-12T09:30:51.055465+0000","last_peered":"2024-09-12T09:30:51.055465+0000","last_clean":"2024-09-12T09:30:51.055465+0000","last_became_active":"2024-09-12T09:30:27.833222+0000","last_became_peered":"2024-09-12T09:30:27.833222+0000","last_unstale":"2024-09-12T09:30:51.055465+0000","last_undegraded":"2024-09-12T09:30:51.055465+0000","last_fullsized":"2024-09-12T09:30:51.055465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:55.948215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":60801771,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":5292,"num_read_kb":89517,"num_write":246,"num_write_kb":44195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":55394924,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"88'261","reported_seq":5413,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064120+0000","last_change":"2024-09-12T09:30:33.671011+0000","last_active":"2024-09-12T09:30:52.064120+0000","last_peered":"2024-09-12T09:30:52.064120+0000","last_clean":"2024-09-12T09:30:52.064120+0000","last_became_active":"2024-09-12T09:30:33.669882+0000","last_became_peered":"2024-09-12T09:30:33.669882+0000","last_unstale":"2024-09-12T09:30:52.064120+0000","last_undegraded":"2024-09-12T09:30:52.064120+0000","last_fullsized":"2024-09-12T09:30:52.064120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:12.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":84519896,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":5560,"num_read_kb":73956,"num_write":345,"num_write_kb":42412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":71454664,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"87'194","reported_seq":1234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049036+0000","last_change":"2024-09-12T09:30:03.657130+0000","last_active":"2024-09-12T09:30:51.049036+0000","last_peered":"2024-09-12T09:30:51.049036+0000","last_clean":"2024-09-12T09:30:51.049036+0000","last_became_active":"2024-09-12T09:30:03.655757+0000","last_became_peered":"2024-09-12T09:30:03.655757+0000","last_unstale":"2024-09-12T09:30:51.049036+0000","last_undegraded":"2024-09-12T09:30:51.049036+0000","last_fullsized":"2024-09-12T09:30:51.049036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:30.005675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":36847756,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":1143,"num_read_kb":11046,"num_write":218,"num_write_kb":22611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":4010639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"87'168","reported_seq":756,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063522+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:30:52.063522+0000","last_peered":"2024-09-12T09:30:52.063522+0000","last_clean":"2024-09-12T09:30:52.063522+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:30:52.063522+0000","last_undegraded":"2024-09-12T09:30:52.063522+0000","last_fullsized":"2024-09-12T09:30:52.063522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:09:07.873122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":50520214,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2518,"num_read_kb":10295,"num_write":189,"num_write_kb":1546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"85'222","reported_seq":10618,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049191+0000","last_change":"2024-09-12T09:30:34.667415+0000","last_active":"2024-09-12T09:30:51.049191+0000","last_peered":"2024-09-12T09:30:51.049191+0000","last_clean":"2024-09-12T09:30:51.049191+0000","last_became_active":"2024-09-12T09:30:34.666472+0000","last_became_peered":"2024-09-12T09:30:34.666472+0000","last_unstale":"2024-09-12T09:30:51.049191+0000","last_undegraded":"2024-09-12T09:30:51.049191+0000","last_fullsized":"2024-09-12T09:30:51.049191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:59:19.922576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":71414865,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":11171,"num_read_kb":95216,"num_write":326,"num_write_kb":54655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":87166181,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"88'112","reported_seq":1216,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063576+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:30:52.063576+0000","last_peered":"2024-09-12T09:30:52.063576+0000","last_clean":"2024-09-12T09:30:52.063576+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:30:52.063576+0000","last_undegraded":"2024-09-12T09:30:52.063576+0000","last_fullsized":"2024-09-12T09:30:52.063576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:42.458299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":37268147,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1487,"num_read_kb":8949,"num_write":201,"num_write_kb":11645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1278804475,"num_objects":1199,"num_object_clones":285,"num_object_copies":2398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1199,"num_whiteouts":476,"num_read":165364,"num_read_kb":1193971,"num_write":8821,"num_write_kb":694598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":345852928,"num_keys_recovered":207,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71372800,"data_stored":1010057241,"data_compressed":9457280,"data_compressed_allocated":63561728,"data_compressed_original":1004849086,"omap_allocated":0,"internal_metadata":0},"log_size":6801,"ondisk_log_size":6801,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":26616,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54276,"kb_used_data":14772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94317564,"statfs":{"total":96636764160,"available":96581185536,"internally_reserved":0,"allocated":15126528,"data_stored":198187261,"data_compressed":1949567,"data_compressed_allocated":12705792,"data_compressed_original":196737426,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":11896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":12181504,"data_stored":145900426,"data_compressed":1616528,"data_compressed_allocated":9728000,"data_compressed_original":144558489,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542167,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49880,"kb_used_data":14528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321960,"statfs":{"total":96636764160,"available":96585687040,"internally_reserved":0,"allocated":14876672,"data_stored":214498996,"data_compressed":1398990,"data_compressed_allocated":12460032,"data_compressed_original":213010299,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38500,"kb_used_data":11476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333340,"statfs":{"total":96636764160,"available":96597340160,"internally_reserved":0,"allocated":11751424,"data_stored":147977815,"data_compressed":1625967,"data_compressed_allocated":9678848,"data_compressed_original":146897436,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42932,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328908,"statfs":{"total":96636764160,"available":96592801792,"internally_reserved":0,"allocated":3510272,"data_stored":26080679,"data_compressed":682259,"data_compressed_allocated":1994752,"data_compressed_original":25309829,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542166,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43080,"kb_used_data":15976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328760,"statfs":{"total":96636764160,"available":96592650240,"internally_reserved":0,"allocated":16359424,"data_stored":243028377,"data_compressed":1617428,"data_compressed_allocated":13795328,"data_compressed_original":241412129,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15294464,"data_stored":241968498,"data_compressed":1610774,"data_compressed_allocated":13758464,"data_compressed_original":240822305,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3252224,"data_stored":33159688,"data_compressed":430613,"data_compressed_allocated":2797568,"data_compressed_original":32897110,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2560000,"data_stored":26135475,"data_compressed":721505,"data_compressed_allocated":2064384,"data_compressed_original":25834117,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":147508323,"data_compressed":1625967,"data_compressed_allocated":9678848,"data_compressed_original":146897436,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11153408,"data_stored":145424760,"data_compressed":1616528,"data_compressed_allocated":9728000,"data_compressed_original":144558489,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13856768,"data_stored":214029521,"data_compressed":1398990,"data_compressed_allocated":12460032,"data_compressed_original":213010299,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":4709750,"data_compressed":109990,"data_compressed_allocated":405504,"data_compressed_original":4681728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14057472,"data_stored":197121226,"data_compressed":1942913,"data_compressed_allocated":12668928,"data_compressed_original":196147602,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:30:54.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:30:54.495 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-12T09:30:54.509 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.509 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:54.540 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:54.540+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:54.641 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:54.640+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:54.647 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.647 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:54.699 INFO:teuthology.orchestra.run.smithi007.stdout:2666 writesame 0x21e308e thru 0x21ed7fe data_size 0x1(0xa771 bytes) 2024-09-12T09:30:54.725 INFO:teuthology.orchestra.run.smithi007.stdout:2667 write 0xc601692 thru 0xc61070b (0xf07a bytes) 2024-09-12T09:30:54.742 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:54.740+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:54.750 INFO:teuthology.orchestra.run.smithi007.stdout:2668 trunc from 0xe854990 to 0x4c0d093 2024-09-12T09:30:54.787 INFO:teuthology.orchestra.run.smithi007.stdout:2669 trunc from 0x4c0d093 to 0x437e176 2024-09-12T09:30:54.797 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.797 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:30:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:30:54.817 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:54.817 INFO:teuthology.orchestra.run.smithi007.stdout:2671 punch from 0x36cd071 to 0x36d7f36, (0xaec5 bytes) 2024-09-12T09:30:54.828 INFO:teuthology.orchestra.run.smithi007.stdout:2672 write 0x477772a thru 0x4785da3 (0xe67a bytes) 2024-09-12T09:30:54.834 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-12T09:30:54.065579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1279394862,"num_objects":1203,"num_object_clones":285,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":476,"num_read":165430,"num_read_kb":1194026,"num_write":8924,"num_write_kb":696228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":347624051,"num_keys_recovered":207,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6876,"ondisk_log_size":6876,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":280260,"kb_used_data":72076,"kb_used_omap":9,"kb_used_meta":207798,"kb_avail":565950780,"statfs":{"total":579820584960,"available":579533598720,"internally_reserved":0,"allocated":73805824,"data_stored":975673554,"data_compressed":8890739,"data_compressed_allocated":60362752,"data_compressed_original":967925608,"omap_allocated":9563,"internal_metadata":212785829},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62019057,"num_objects":74,"num_object_clones":35,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":11,"num_read":4366,"num_read_kb":46492,"num_write":767,"num_write_kb":13958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.071719"},"pg_stats":[{"pgid":"3.1f","version":"86'417","reported_seq":4869,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049896+0000","last_change":"2024-09-12T09:29:58.167241+0000","last_active":"2024-09-12T09:30:51.049896+0000","last_peered":"2024-09-12T09:30:51.049896+0000","last_clean":"2024-09-12T09:30:51.049896+0000","last_became_active":"2024-09-12T09:29:58.165903+0000","last_became_peered":"2024-09-12T09:29:58.165903+0000","last_unstale":"2024-09-12T09:30:51.049896+0000","last_undegraded":"2024-09-12T09:30:51.049896+0000","last_fullsized":"2024-09-12T09:30:51.049896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:36:31.153821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":86565125,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":5570,"num_read_kb":88595,"num_write":585,"num_write_kb":45291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":4329673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"87'403","reported_seq":4096,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052342+0000","last_change":"2024-09-12T09:30:01.194793+0000","last_active":"2024-09-12T09:30:51.052342+0000","last_peered":"2024-09-12T09:30:51.052342+0000","last_clean":"2024-09-12T09:30:51.052342+0000","last_became_active":"2024-09-12T09:30:01.169430+0000","last_became_peered":"2024-09-12T09:30:01.169430+0000","last_unstale":"2024-09-12T09:30:51.052342+0000","last_undegraded":"2024-09-12T09:30:51.052342+0000","last_fullsized":"2024-09-12T09:30:51.052342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:46.408572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":41789349,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":5693,"num_read_kb":33945,"num_write":558,"num_write_kb":33688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":352930,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"82'162","reported_seq":3649,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055729+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:30:51.055729+0000","last_peered":"2024-09-12T09:30:51.055729+0000","last_clean":"2024-09-12T09:30:51.055729+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:30:51.055729+0000","last_undegraded":"2024-09-12T09:30:51.055729+0000","last_fullsized":"2024-09-12T09:30:51.055729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:27:02.526323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":43645672,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":20,"num_read":3818,"num_read_kb":65645,"num_write":144,"num_write_kb":35929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"87'179","reported_seq":3874,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055407+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:30:51.055407+0000","last_peered":"2024-09-12T09:30:51.055407+0000","last_clean":"2024-09-12T09:30:51.055407+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:30:51.055407+0000","last_undegraded":"2024-09-12T09:30:51.055407+0000","last_fullsized":"2024-09-12T09:30:51.055407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:32.316709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":19972439,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4199,"num_read_kb":33781,"num_write":196,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"87'169","reported_seq":4815,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049540+0000","last_change":"2024-09-12T09:29:55.231536+0000","last_active":"2024-09-12T09:30:51.049540+0000","last_peered":"2024-09-12T09:30:51.049540+0000","last_clean":"2024-09-12T09:30:51.049540+0000","last_became_active":"2024-09-12T09:29:55.137238+0000","last_became_peered":"2024-09-12T09:29:55.137238+0000","last_unstale":"2024-09-12T09:30:51.049540+0000","last_undegraded":"2024-09-12T09:30:51.049540+0000","last_fullsized":"2024-09-12T09:30:51.049540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:50.017859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":27142706,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":5284,"num_read_kb":12908,"num_write":179,"num_write_kb":4190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1206936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"87'200","reported_seq":2321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053098+0000","last_change":"2024-09-12T09:30:46.979355+0000","last_active":"2024-09-12T09:30:52.053098+0000","last_peered":"2024-09-12T09:30:52.053098+0000","last_clean":"2024-09-12T09:30:52.053098+0000","last_became_active":"2024-09-12T09:30:46.978472+0000","last_became_peered":"2024-09-12T09:30:46.978472+0000","last_unstale":"2024-09-12T09:30:52.053098+0000","last_undegraded":"2024-09-12T09:30:52.053098+0000","last_fullsized":"2024-09-12T09:30:52.053098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:44.155070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":9197978,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2365,"num_read_kb":9518,"num_write":204,"num_write_kb":30034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2195003,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"87'108","reported_seq":711,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052000+0000","last_change":"2024-09-12T09:30:44.080862+0000","last_active":"2024-09-12T09:30:51.052000+0000","last_peered":"2024-09-12T09:30:51.052000+0000","last_clean":"2024-09-12T09:30:51.052000+0000","last_became_active":"2024-09-12T09:30:43.967238+0000","last_became_peered":"2024-09-12T09:30:43.967238+0000","last_unstale":"2024-09-12T09:30:51.052000+0000","last_undegraded":"2024-09-12T09:30:51.052000+0000","last_fullsized":"2024-09-12T09:30:51.052000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:29.283242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":24759249,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":641,"num_read_kb":4170,"num_write":92,"num_write_kb":1812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":970649,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"87'141","reported_seq":1291,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.051983+0000","last_change":"2024-09-12T09:29:55.177304+0000","last_active":"2024-09-12T09:30:51.051983+0000","last_peered":"2024-09-12T09:30:51.051983+0000","last_clean":"2024-09-12T09:30:51.051983+0000","last_became_active":"2024-09-12T09:29:55.134985+0000","last_became_peered":"2024-09-12T09:29:55.134985+0000","last_unstale":"2024-09-12T09:30:51.051983+0000","last_undegraded":"2024-09-12T09:30:51.051983+0000","last_fullsized":"2024-09-12T09:30:51.051983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:23.074794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":11573008,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1170,"num_read_kb":5400,"num_write":151,"num_write_kb":4735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":524806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"87'130","reported_seq":2461,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049763+0000","last_change":"2024-09-12T09:29:55.167682+0000","last_active":"2024-09-12T09:30:51.049763+0000","last_peered":"2024-09-12T09:30:51.049763+0000","last_clean":"2024-09-12T09:30:51.049763+0000","last_became_active":"2024-09-12T09:29:55.136789+0000","last_became_peered":"2024-09-12T09:29:55.136789+0000","last_unstale":"2024-09-12T09:30:51.049763+0000","last_undegraded":"2024-09-12T09:30:51.049763+0000","last_fullsized":"2024-09-12T09:30:51.049763+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:10:27.604010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":42465040,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":2648,"num_read_kb":47032,"num_write":229,"num_write_kb":36221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8413,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"88'226","reported_seq":13691,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055728+0000","last_change":"2024-09-12T09:30:33.669800+0000","last_active":"2024-09-12T09:30:51.055728+0000","last_peered":"2024-09-12T09:30:51.055728+0000","last_clean":"2024-09-12T09:30:51.055728+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:30:51.055728+0000","last_undegraded":"2024-09-12T09:30:51.055728+0000","last_fullsized":"2024-09-12T09:30:51.055728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:00.826841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":63441202,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":13510,"num_read_kb":69961,"num_write":304,"num_write_kb":27133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'482","reported_seq":5563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064643+0000","last_change":"2024-09-12T09:30:34.672921+0000","last_active":"2024-09-12T09:30:52.064643+0000","last_peered":"2024-09-12T09:30:52.064643+0000","last_clean":"2024-09-12T09:30:52.064643+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:30:52.064643+0000","last_undegraded":"2024-09-12T09:30:52.064643+0000","last_fullsized":"2024-09-12T09:30:52.064643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:33:38.199970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":83238189,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":22,"num_read":6392,"num_read_kb":77369,"num_write":738,"num_write_kb":42344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"89'524","reported_seq":8201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053674+0000","last_change":"2024-09-12T09:30:33.668610+0000","last_active":"2024-09-12T09:30:52.053674+0000","last_peered":"2024-09-12T09:30:52.053674+0000","last_clean":"2024-09-12T09:30:52.053674+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:30:52.053674+0000","last_undegraded":"2024-09-12T09:30:52.053674+0000","last_fullsized":"2024-09-12T09:30:52.053674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:00.155518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":22543438,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":9525,"num_read_kb":20252,"num_write":863,"num_write_kb":23530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064044+0000","last_change":"2024-09-12T09:30:30.871108+0000","last_active":"2024-09-12T09:30:52.064044+0000","last_peered":"2024-09-12T09:30:52.064044+0000","last_clean":"2024-09-12T09:30:52.064044+0000","last_became_active":"2024-09-12T09:30:30.870346+0000","last_became_peered":"2024-09-12T09:30:30.870346+0000","last_unstale":"2024-09-12T09:30:52.064044+0000","last_undegraded":"2024-09-12T09:30:52.064044+0000","last_fullsized":"2024-09-12T09:30:52.064044+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:23.248076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"85'122","reported_seq":4862,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049410+0000","last_change":"2024-09-12T09:30:46.982993+0000","last_active":"2024-09-12T09:30:51.049410+0000","last_peered":"2024-09-12T09:30:51.049410+0000","last_clean":"2024-09-12T09:30:51.049410+0000","last_became_active":"2024-09-12T09:30:46.981203+0000","last_became_peered":"2024-09-12T09:30:46.981203+0000","last_unstale":"2024-09-12T09:30:51.049410+0000","last_undegraded":"2024-09-12T09:30:51.049410+0000","last_fullsized":"2024-09-12T09:30:51.049410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:06.188499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":21006461,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":5201,"num_read_kb":19833,"num_write":107,"num_write_kb":4869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":4829848,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053870+0000","last_change":"2024-09-12T09:30:43.963886+0000","last_active":"2024-09-12T09:30:52.053870+0000","last_peered":"2024-09-12T09:30:52.053870+0000","last_clean":"2024-09-12T09:30:52.053870+0000","last_became_active":"2024-09-12T09:30:43.962849+0000","last_became_peered":"2024-09-12T09:30:43.962849+0000","last_unstale":"2024-09-12T09:30:52.053870+0000","last_undegraded":"2024-09-12T09:30:52.053870+0000","last_fullsized":"2024-09-12T09:30:52.053870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:38.942689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"85'138","reported_seq":2041,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063785+0000","last_change":"2024-09-12T09:30:33.673235+0000","last_active":"2024-09-12T09:30:52.063785+0000","last_peered":"2024-09-12T09:30:52.063785+0000","last_clean":"2024-09-12T09:30:52.063785+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:30:52.063785+0000","last_undegraded":"2024-09-12T09:30:52.063785+0000","last_fullsized":"2024-09-12T09:30:52.063785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:12:08.342915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":25079818,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":12,"num_read":1788,"num_read_kb":14526,"num_write":197,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064285+0000","last_change":"2024-09-12T09:30:30.875450+0000","last_active":"2024-09-12T09:30:52.064285+0000","last_peered":"2024-09-12T09:30:52.064285+0000","last_clean":"2024-09-12T09:30:52.064285+0000","last_became_active":"2024-09-12T09:30:30.874655+0000","last_became_peered":"2024-09-12T09:30:30.874655+0000","last_unstale":"2024-09-12T09:30:52.064285+0000","last_undegraded":"2024-09-12T09:30:52.064285+0000","last_fullsized":"2024-09-12T09:30:52.064285+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:30.720674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"87'140","reported_seq":4852,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049566+0000","last_change":"2024-09-12T09:30:46.985105+0000","last_active":"2024-09-12T09:30:51.049566+0000","last_peered":"2024-09-12T09:30:51.049566+0000","last_clean":"2024-09-12T09:30:51.049566+0000","last_became_active":"2024-09-12T09:30:46.983676+0000","last_became_peered":"2024-09-12T09:30:46.983676+0000","last_unstale":"2024-09-12T09:30:51.049566+0000","last_undegraded":"2024-09-12T09:30:51.049566+0000","last_fullsized":"2024-09-12T09:30:51.049566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:18.469021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":24332273,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":4487,"num_read_kb":20388,"num_write":222,"num_write_kb":43382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":7432750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049288+0000","last_change":"2024-09-12T09:30:43.961453+0000","last_active":"2024-09-12T09:30:51.049288+0000","last_peered":"2024-09-12T09:30:51.049288+0000","last_clean":"2024-09-12T09:30:51.049288+0000","last_became_active":"2024-09-12T09:30:43.960994+0000","last_became_peered":"2024-09-12T09:30:43.960994+0000","last_unstale":"2024-09-12T09:30:51.049288+0000","last_undegraded":"2024-09-12T09:30:51.049288+0000","last_fullsized":"2024-09-12T09:30:51.049288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.668460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"87'133","reported_seq":13344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055958+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:30:51.055958+0000","last_peered":"2024-09-12T09:30:51.055958+0000","last_clean":"2024-09-12T09:30:51.055958+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:30:51.055958+0000","last_undegraded":"2024-09-12T09:30:51.055958+0000","last_fullsized":"2024-09-12T09:30:51.055958+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:35:50.066462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":35639524,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":13338,"num_read_kb":55483,"num_write":116,"num_write_kb":11232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"88'323","reported_seq":4256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053805+0000","last_change":"2024-09-12T09:29:55.204077+0000","last_active":"2024-09-12T09:30:52.053805+0000","last_peered":"2024-09-12T09:30:52.053805+0000","last_clean":"2024-09-12T09:30:52.053805+0000","last_became_active":"2024-09-12T09:29:55.135753+0000","last_became_peered":"2024-09-12T09:29:55.135753+0000","last_unstale":"2024-09-12T09:30:52.053805+0000","last_undegraded":"2024-09-12T09:30:52.053805+0000","last_fullsized":"2024-09-12T09:30:52.053805+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:52.900351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":25327632,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":5188,"num_read_kb":19677,"num_write":397,"num_write_kb":9398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":57888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":165,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055184+0000","last_change":"2024-09-12T09:30:43.969129+0000","last_active":"2024-09-12T09:30:51.055184+0000","last_peered":"2024-09-12T09:30:51.055184+0000","last_clean":"2024-09-12T09:30:51.055184+0000","last_became_active":"2024-09-12T09:30:43.962175+0000","last_became_peered":"2024-09-12T09:30:43.962175+0000","last_unstale":"2024-09-12T09:30:51.055184+0000","last_undegraded":"2024-09-12T09:30:51.055184+0000","last_fullsized":"2024-09-12T09:30:51.055184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:00:15.922849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"87'169","reported_seq":3503,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.048932+0000","last_change":"2024-09-12T09:30:46.984392+0000","last_active":"2024-09-12T09:30:51.048932+0000","last_peered":"2024-09-12T09:30:51.048932+0000","last_clean":"2024-09-12T09:30:51.048932+0000","last_became_active":"2024-09-12T09:30:46.982399+0000","last_became_peered":"2024-09-12T09:30:46.982399+0000","last_unstale":"2024-09-12T09:30:51.048932+0000","last_undegraded":"2024-09-12T09:30:51.048932+0000","last_fullsized":"2024-09-12T09:30:51.048932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:24.471861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":45358330,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":3125,"num_read_kb":34218,"num_write":142,"num_write_kb":14520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2593871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":154,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049222+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:30:51.049222+0000","last_peered":"2024-09-12T09:30:51.049222+0000","last_clean":"2024-09-12T09:30:51.049222+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:30:51.049222+0000","last_undegraded":"2024-09-12T09:30:51.049222+0000","last_fullsized":"2024-09-12T09:30:51.049222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"87'181","reported_seq":9732,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.048863+0000","last_change":"2024-09-12T09:29:55.163561+0000","last_active":"2024-09-12T09:30:51.048863+0000","last_peered":"2024-09-12T09:30:51.048863+0000","last_clean":"2024-09-12T09:30:51.048863+0000","last_became_active":"2024-09-12T09:29:55.134480+0000","last_became_peered":"2024-09-12T09:29:55.134480+0000","last_unstale":"2024-09-12T09:30:51.048863+0000","last_undegraded":"2024-09-12T09:30:51.048863+0000","last_fullsized":"2024-09-12T09:30:51.048863+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:05.743583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":38984553,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":10443,"num_read_kb":32779,"num_write":159,"num_write_kb":20082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2206098,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063507+0000","last_change":"2024-09-12T09:30:43.963299+0000","last_active":"2024-09-12T09:30:52.063507+0000","last_peered":"2024-09-12T09:30:52.063507+0000","last_clean":"2024-09-12T09:30:52.063507+0000","last_became_active":"2024-09-12T09:30:43.961820+0000","last_became_peered":"2024-09-12T09:30:43.961820+0000","last_unstale":"2024-09-12T09:30:52.063507+0000","last_undegraded":"2024-09-12T09:30:52.063507+0000","last_fullsized":"2024-09-12T09:30:52.063507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:20.604956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"87'122","reported_seq":4617,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055267+0000","last_change":"2024-09-12T09:30:12.711941+0000","last_active":"2024-09-12T09:30:51.055267+0000","last_peered":"2024-09-12T09:30:51.055267+0000","last_clean":"2024-09-12T09:30:51.055267+0000","last_became_active":"2024-09-12T09:30:12.711307+0000","last_became_peered":"2024-09-12T09:30:12.711307+0000","last_unstale":"2024-09-12T09:30:51.055267+0000","last_undegraded":"2024-09-12T09:30:51.055267+0000","last_fullsized":"2024-09-12T09:30:51.055267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:23.436691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":32879263,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":5336,"num_read_kb":41925,"num_write":139,"num_write_kb":10668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":154,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055294+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:30:51.055294+0000","last_peered":"2024-09-12T09:30:51.055294+0000","last_clean":"2024-09-12T09:30:51.055294+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:30:51.055294+0000","last_undegraded":"2024-09-12T09:30:51.055294+0000","last_fullsized":"2024-09-12T09:30:51.055294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063245+0000","last_change":"2024-09-12T09:30:30.889337+0000","last_active":"2024-09-12T09:30:52.063245+0000","last_peered":"2024-09-12T09:30:52.063245+0000","last_clean":"2024-09-12T09:30:52.063245+0000","last_became_active":"2024-09-12T09:30:30.873745+0000","last_became_peered":"2024-09-12T09:30:30.873745+0000","last_unstale":"2024-09-12T09:30:52.063245+0000","last_undegraded":"2024-09-12T09:30:52.063245+0000","last_fullsized":"2024-09-12T09:30:52.063245+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:57.848261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"85'232","reported_seq":3904,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.051982+0000","last_change":"2024-09-12T09:30:46.981798+0000","last_active":"2024-09-12T09:30:51.051982+0000","last_peered":"2024-09-12T09:30:51.051982+0000","last_clean":"2024-09-12T09:30:51.051982+0000","last_became_active":"2024-09-12T09:30:46.980108+0000","last_became_peered":"2024-09-12T09:30:46.980108+0000","last_unstale":"2024-09-12T09:30:51.051982+0000","last_undegraded":"2024-09-12T09:30:51.051982+0000","last_fullsized":"2024-09-12T09:30:51.051982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:57:43.658520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":38826208,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":20,"num_read":5334,"num_read_kb":49163,"num_write":279,"num_write_kb":20701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":10746792,"num_keys_recovered":21,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"86'179","reported_seq":2078,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053350+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:30:52.053350+0000","last_peered":"2024-09-12T09:30:52.053350+0000","last_clean":"2024-09-12T09:30:52.053350+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:30:52.053350+0000","last_undegraded":"2024-09-12T09:30:52.053350+0000","last_fullsized":"2024-09-12T09:30:52.053350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:10.027724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":22133403,"num_objects":33,"num_object_clones":4,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":1974,"num_read_kb":11122,"num_write":193,"num_write_kb":8073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'178","reported_seq":4786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.053015+0000","last_change":"2024-09-12T09:30:46.979031+0000","last_active":"2024-09-12T09:30:52.053015+0000","last_peered":"2024-09-12T09:30:52.053015+0000","last_clean":"2024-09-12T09:30:52.053015+0000","last_became_active":"2024-09-12T09:30:46.977847+0000","last_became_peered":"2024-09-12T09:30:46.977847+0000","last_unstale":"2024-09-12T09:30:52.053015+0000","last_undegraded":"2024-09-12T09:30:52.053015+0000","last_fullsized":"2024-09-12T09:30:52.053015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:10.148048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":28661649,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":4680,"num_read_kb":11083,"num_write":254,"num_write_kb":5580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":7309669,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'268","reported_seq":5533,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.052401+0000","last_change":"2024-09-12T09:29:55.239142+0000","last_active":"2024-09-12T09:30:51.052401+0000","last_peered":"2024-09-12T09:30:51.052401+0000","last_clean":"2024-09-12T09:30:51.052401+0000","last_became_active":"2024-09-12T09:29:55.135093+0000","last_became_peered":"2024-09-12T09:29:55.135093+0000","last_unstale":"2024-09-12T09:30:51.052401+0000","last_undegraded":"2024-09-12T09:30:51.052401+0000","last_fullsized":"2024-09-12T09:30:51.052401+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:48:40.796288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":73510413,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":5875,"num_read_kb":70738,"num_write":450,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1338871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"87'155","reported_seq":2253,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055665+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:30:51.055665+0000","last_peered":"2024-09-12T09:30:51.055665+0000","last_clean":"2024-09-12T09:30:51.055665+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:30:51.055665+0000","last_undegraded":"2024-09-12T09:30:51.055665+0000","last_fullsized":"2024-09-12T09:30:51.055665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:04.790845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":26022144,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3192,"num_read_kb":30963,"num_write":178,"num_write_kb":16002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"88'157","reported_seq":7209,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063799+0000","last_change":"2024-09-12T09:30:33.675916+0000","last_active":"2024-09-12T09:30:52.063799+0000","last_peered":"2024-09-12T09:30:52.063799+0000","last_clean":"2024-09-12T09:30:52.063799+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:30:52.063799+0000","last_undegraded":"2024-09-12T09:30:52.063799+0000","last_fullsized":"2024-09-12T09:30:52.063799+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:32.494075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":23336760,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":7417,"num_read_kb":24518,"num_write":220,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"87'206","reported_seq":4687,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.055465+0000","last_change":"2024-09-12T09:30:27.834395+0000","last_active":"2024-09-12T09:30:51.055465+0000","last_peered":"2024-09-12T09:30:51.055465+0000","last_clean":"2024-09-12T09:30:51.055465+0000","last_became_active":"2024-09-12T09:30:27.833222+0000","last_became_peered":"2024-09-12T09:30:27.833222+0000","last_unstale":"2024-09-12T09:30:51.055465+0000","last_undegraded":"2024-09-12T09:30:51.055465+0000","last_fullsized":"2024-09-12T09:30:51.055465+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:55.948215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":60801771,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":5292,"num_read_kb":89517,"num_write":246,"num_write_kb":44195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":55394924,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"88'261","reported_seq":5413,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.064120+0000","last_change":"2024-09-12T09:30:33.671011+0000","last_active":"2024-09-12T09:30:52.064120+0000","last_peered":"2024-09-12T09:30:52.064120+0000","last_clean":"2024-09-12T09:30:52.064120+0000","last_became_active":"2024-09-12T09:30:33.669882+0000","last_became_peered":"2024-09-12T09:30:33.669882+0000","last_unstale":"2024-09-12T09:30:52.064120+0000","last_undegraded":"2024-09-12T09:30:52.064120+0000","last_fullsized":"2024-09-12T09:30:52.064120+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:12.175815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":84519896,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":5560,"num_read_kb":73956,"num_write":345,"num_write_kb":42412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":71454664,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"87'194","reported_seq":1234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049036+0000","last_change":"2024-09-12T09:30:03.657130+0000","last_active":"2024-09-12T09:30:51.049036+0000","last_peered":"2024-09-12T09:30:51.049036+0000","last_clean":"2024-09-12T09:30:51.049036+0000","last_became_active":"2024-09-12T09:30:03.655757+0000","last_became_peered":"2024-09-12T09:30:03.655757+0000","last_unstale":"2024-09-12T09:30:51.049036+0000","last_undegraded":"2024-09-12T09:30:51.049036+0000","last_fullsized":"2024-09-12T09:30:51.049036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:10:30.005675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":36847756,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":1143,"num_read_kb":11046,"num_write":218,"num_write_kb":22611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":4010639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"87'168","reported_seq":756,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063522+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:30:52.063522+0000","last_peered":"2024-09-12T09:30:52.063522+0000","last_clean":"2024-09-12T09:30:52.063522+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:30:52.063522+0000","last_undegraded":"2024-09-12T09:30:52.063522+0000","last_fullsized":"2024-09-12T09:30:52.063522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:09:07.873122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":50520214,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2518,"num_read_kb":10295,"num_write":189,"num_write_kb":1546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"85'222","reported_seq":10618,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-12T09:30:51.049191+0000","last_change":"2024-09-12T09:30:34.667415+0000","last_active":"2024-09-12T09:30:51.049191+0000","last_peered":"2024-09-12T09:30:51.049191+0000","last_clean":"2024-09-12T09:30:51.049191+0000","last_became_active":"2024-09-12T09:30:34.666472+0000","last_became_peered":"2024-09-12T09:30:34.666472+0000","last_unstale":"2024-09-12T09:30:51.049191+0000","last_undegraded":"2024-09-12T09:30:51.049191+0000","last_fullsized":"2024-09-12T09:30:51.049191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:59:19.922576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":71414865,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":11171,"num_read_kb":95216,"num_write":326,"num_write_kb":54655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":87166181,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"88'112","reported_seq":1216,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-12T09:30:52.063576+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:30:52.063576+0000","last_peered":"2024-09-12T09:30:52.063576+0000","last_clean":"2024-09-12T09:30:52.063576+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:30:52.063576+0000","last_undegraded":"2024-09-12T09:30:52.063576+0000","last_fullsized":"2024-09-12T09:30:52.063576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:01:42.458299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":37268147,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1487,"num_read_kb":8949,"num_write":201,"num_write_kb":11645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1278804475,"num_objects":1199,"num_object_clones":285,"num_object_copies":2398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1199,"num_whiteouts":476,"num_read":165364,"num_read_kb":1193971,"num_write":8821,"num_write_kb":694598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":345852928,"num_keys_recovered":207,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71372800,"data_stored":1010057241,"data_compressed":9457280,"data_compressed_allocated":63561728,"data_compressed_original":1004849086,"omap_allocated":0,"internal_metadata":0},"log_size":6801,"ondisk_log_size":6801,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":26616,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54276,"kb_used_data":14772,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94317564,"statfs":{"total":96636764160,"available":96581185536,"internally_reserved":0,"allocated":15126528,"data_stored":198187261,"data_compressed":1949567,"data_compressed_allocated":12705792,"data_compressed_original":196737426,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":11896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":12181504,"data_stored":145900426,"data_compressed":1616528,"data_compressed_allocated":9728000,"data_compressed_original":144558489,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542167,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49880,"kb_used_data":14528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321960,"statfs":{"total":96636764160,"available":96585687040,"internally_reserved":0,"allocated":14876672,"data_stored":214498996,"data_compressed":1398990,"data_compressed_allocated":12460032,"data_compressed_original":213010299,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38500,"kb_used_data":11476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333340,"statfs":{"total":96636764160,"available":96597340160,"internally_reserved":0,"allocated":11751424,"data_stored":147977815,"data_compressed":1625967,"data_compressed_allocated":9678848,"data_compressed_original":146897436,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42932,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328908,"statfs":{"total":96636764160,"available":96592801792,"internally_reserved":0,"allocated":3510272,"data_stored":26080679,"data_compressed":682259,"data_compressed_allocated":1994752,"data_compressed_original":25309829,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542166,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43080,"kb_used_data":15976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328760,"statfs":{"total":96636764160,"available":96592650240,"internally_reserved":0,"allocated":16359424,"data_stored":243028377,"data_compressed":1617428,"data_compressed_allocated":13795328,"data_compressed_original":241412129,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15294464,"data_stored":241968498,"data_compressed":1610774,"data_compressed_allocated":13758464,"data_compressed_original":240822305,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3252224,"data_stored":33159688,"data_compressed":430613,"data_compressed_allocated":2797568,"data_compressed_original":32897110,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2560000,"data_stored":26135475,"data_compressed":721505,"data_compressed_allocated":2064384,"data_compressed_original":25834117,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10731520,"data_stored":147508323,"data_compressed":1625967,"data_compressed_allocated":9678848,"data_compressed_original":146897436,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11153408,"data_stored":145424760,"data_compressed":1616528,"data_compressed_allocated":9728000,"data_compressed_original":144558489,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13856768,"data_stored":214029521,"data_compressed":1398990,"data_compressed_allocated":12460032,"data_compressed_original":213010299,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":4709750,"data_compressed":109990,"data_compressed_allocated":405504,"data_compressed_original":4681728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14057472,"data_stored":197121226,"data_compressed":1942913,"data_compressed_allocated":12668928,"data_compressed_original":196147602,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:30:54.836 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T09:30:54.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:30:54.836 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-12T09:30:54.836 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T09:30:54.836 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T09:30:54.837 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-12T09:30:54.855 INFO:teuthology.orchestra.run.smithi007.stdout:2673 writesame 0x73cf05f thru 0x73d9c4a data_size 0x2(0xabec bytes) 2024-09-12T09:30:54.855 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:54.860 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:54.859+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:54.873 INFO:teuthology.orchestra.run.smithi007.stdout:2676 writesame 0xb605843 thru 0xb60e41e data_size 0x2(0x8bdc bytes) 2024-09-12T09:30:54.873 INFO:teuthology.orchestra.run.smithi007.stdout:2677 write 0x986e3ac thru 0x9870355 (0x1faa bytes) 2024-09-12T09:30:54.879 INFO:teuthology.orchestra.run.smithi007.stdout:2678 punch from 0x18042e1 to 0x18083c2, (0x40e1 bytes) 2024-09-12T09:30:54.879 INFO:teuthology.orchestra.run.smithi007.stdout:2679 punch from 0x890b464 to 0x890f549, (0x40e5 bytes) 2024-09-12T09:30:54.880 INFO:teuthology.orchestra.run.smithi007.stdout:2680 read 0x1a3021 thru 0x1b05c0 (0xd5a0 bytes) 2024-09-12T09:30:54.881 INFO:teuthology.orchestra.run.smithi007.stdout:2682 write 0x859753f thru 0x859a9c3 (0x3485 bytes) 2024-09-12T09:30:54.891 INFO:teuthology.orchestra.run.smithi007.stdout:2683 trunc from 0xb60e41f to 0x9f61d89 2024-09-12T09:30:54.922 INFO:teuthology.orchestra.run.smithi007.stdout:2685 writesame 0xddf2cf4 thru 0xddf3ddb data_size 0x4(0x10e8 bytes) 2024-09-12T09:30:54.922 INFO:teuthology.orchestra.run.smithi007.stdout:2686 read 0x97b32bf thru 0x97b9fa7 (0x6ce9 bytes) 2024-09-12T09:30:54.932 INFO:teuthology.orchestra.run.smithi007.stdout:2687 write 0xe0c1545 thru 0xe0c9fef (0x8aab bytes) 2024-09-12T09:30:54.940 INFO:teuthology.orchestra.run.smithi007.stdout:2688 write 0x9351e6a thru 0x935bb0a (0x9ca1 bytes) 2024-09-12T09:30:54.942 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:54.939+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:54.947 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:54.947 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:54.953 INFO:teuthology.orchestra.run.smithi007.stdout:2689 read 0xd2cdada thru 0xd2d874b (0xac72 bytes) 2024-09-12T09:30:54.953 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:54.953 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:54.953 INFO:teuthology.orchestra.run.smithi007.stdout:2694 write 0x9a722c2 thru 0x9a7daec (0xb82b bytes) 2024-09-12T09:30:54.964 INFO:teuthology.orchestra.run.smithi007.stdout:2697 read 0xacdcec5 thru 0xacde177 (0x12b3 bytes) 2024-09-12T09:30:54.966 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:54.966 INFO:teuthology.orchestra.run.smithi007.stdout:2698 trunc from 0xe0c9ff0 to 0xd97b951 2024-09-12T09:30:54.981 INFO:teuthology.orchestra.run.smithi007.stdout:2699 read 0xc5be80f thru 0xc5c01c1 (0x19b3 bytes) 2024-09-12T09:30:54.981 INFO:teuthology.orchestra.run.smithi007.stdout:2700 read 0x4afbef0 thru 0x4afcbb2 (0xcc3 bytes) 2024-09-12T09:30:54.981 INFO:teuthology.orchestra.run.smithi007.stdout:2701 punch from 0x8353483 to 0x8362ba1, (0xf71e bytes) 2024-09-12T09:30:54.986 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:54.986 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T09:30:54.986 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:55.013 INFO:teuthology.orchestra.run.smithi007.stdout:2702 trunc from 0xd97b951 to 0x1e72d18 2024-09-12T09:30:55.043 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:55.040+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:55.053 INFO:teuthology.orchestra.run.smithi007.stdout:2703 trunc from 0x1e72d18 to 0x33011af 2024-09-12T09:30:55.063 INFO:teuthology.orchestra.run.smithi007.stdout:2704 write 0x117807e thru 0x117b3bf (0x3342 bytes) 2024-09-12T09:30:55.070 INFO:teuthology.orchestra.run.smithi007.stdout:2705 read 0x21fa8e4 thru 0x2205385 (0xaaa2 bytes) 2024-09-12T09:30:55.070 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.096 INFO:teuthology.orchestra.run.smithi007.stdout:2708 write 0xa72a1a1 thru 0xa7359e9 (0xb849 bytes) 2024-09-12T09:30:55.110 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:55.110 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:55.110 INFO:teuthology.orchestra.run.smithi007.stdout:2709 read 0xa434781 thru 0xa437025 (0x28a5 bytes) 2024-09-12T09:30:55.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:55.133 INFO:teuthology.orchestra.run.smithi007.stdout:2710 trunc from 0xa7359ea to 0xaa7c92 2024-09-12T09:30:55.144 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:55.140+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:55.208 INFO:teuthology.orchestra.run.smithi007.stdout:2711 writesame 0xe61427c thru 0xe6222f6 data_size 0x1(0xe07b bytes) 2024-09-12T09:30:55.208 INFO:teuthology.orchestra.run.smithi007.stdout:2713 read 0x52aeaed thru 0x52af44d (0x961 bytes) 2024-09-12T09:30:55.208 INFO:teuthology.orchestra.run.smithi007.stdout:2716 writesame 0x5a93fcd thru 0x5a96108 data_size 0x3(0x213c bytes) 2024-09-12T09:30:55.231 INFO:teuthology.orchestra.run.smithi007.stdout:2718 writesame 0xee1c4d1 thru 0xee25cf7 data_size 0x1(0x9827 bytes) 2024-09-12T09:30:55.231 INFO:teuthology.orchestra.run.smithi007.stdout:2719 write 0x3b95fd3 thru 0x3b9b818 (0x5846 bytes) 2024-09-12T09:30:55.239 INFO:teuthology.orchestra.run.smithi007.stdout:2720 punch from 0x2d16cb7 to 0x2d1b340, (0x4689 bytes) 2024-09-12T09:30:55.245 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:55.243+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:55.253 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:55.253 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:55.258 INFO:teuthology.orchestra.run.smithi007.stdout:2721 trunc from 0xee25cf8 to 0x7c5b032 2024-09-12T09:30:55.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:55.271 INFO:teuthology.orchestra.run.smithi007.stdout:2724 writesame 0x6223e9f thru 0x6232861 data_size 0x53(0xe9c3 bytes) 2024-09-12T09:30:55.294 INFO:teuthology.orchestra.run.smithi007.stdout:2727 trunc from 0x7c5b032 to 0x5112b9c 2024-09-12T09:30:55.332 INFO:teuthology.orchestra.run.smithi007.stdout:2728 write 0x7f1ac0f thru 0x7f1ed35 (0x4127 bytes) 2024-09-12T09:30:55.339 INFO:teuthology.orchestra.run.smithi007.stdout:2729 read 0x40bfee thru 0x414106 (0x8119 bytes) 2024-09-12T09:30:55.340 INFO:teuthology.orchestra.run.smithi007.stdout:2730 write 0x5814acc thru 0x58186d0 (0x3c05 bytes) 2024-09-12T09:30:55.346 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:55.344+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:55.349 INFO:teuthology.orchestra.run.smithi007.stdout:2731 write 0x5c08f78 thru 0x5c142e0 (0xb369 bytes) 2024-09-12T09:30:55.365 INFO:teuthology.orchestra.run.smithi007.stdout:2732 write 0x2be164c thru 0x2be90b0 (0x7a65 bytes) 2024-09-12T09:30:55.378 INFO:teuthology.orchestra.run.smithi007.stdout:2733 punch from 0x2d40db4 to 0x2d45617, (0x4863 bytes) 2024-09-12T09:30:55.379 INFO:teuthology.orchestra.run.smithi007.stdout:2734 read 0xd8e797 thru 0xd8f261 (0xacb bytes) 2024-09-12T09:30:55.379 INFO:teuthology.orchestra.run.smithi007.stdout:2735 writesame 0x5c269fb thru 0x5c32abb data_size 0x47(0xc0c1 bytes) 2024-09-12T09:30:55.418 INFO:teuthology.orchestra.run.smithi007.stdout:2736 writesame 0xeb85bb8 thru 0xeb8c106 data_size 0x7(0x654f bytes) 2024-09-12T09:30:55.418 INFO:teuthology.orchestra.run.smithi007.stdout:2737 write 0x9974e04 thru 0x997562b (0x828 bytes) 2024-09-12T09:30:55.426 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.426 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.426 INFO:teuthology.orchestra.run.smithi007.stdout:2743 read 0x65a9339 thru 0x65acae0 (0x37a8 bytes) 2024-09-12T09:30:55.426 INFO:teuthology.orchestra.run.smithi007.stdout:2745 write 0x1f067f0 thru 0x1f12ab4 (0xc2c5 bytes) 2024-09-12T09:30:55.440 INFO:teuthology.orchestra.run.smithi007.stdout:2746 writesame 0xa20b162 thru 0xa20b743 data_size 0x3(0x5e2 bytes) 2024-09-12T09:30:55.447 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:55.443+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:55.447 INFO:teuthology.orchestra.run.smithi007.stdout:2748 write 0x16f7d86 thru 0x16fde7e (0x60f9 bytes) 2024-09-12T09:30:55.448 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:55.449 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:55.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:30:55.485 INFO:teuthology.orchestra.run.smithi007.stdout:2749 trunc from 0xeb8c107 to 0x48823eb 2024-09-12T09:30:55.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:30:55.543 INFO:teuthology.orchestra.run.smithi007.stdout:2750 writesame 0xde7119f thru 0xde7bc73 data_size 0x1(0xaad5 bytes) 2024-09-12T09:30:55.543 INFO:teuthology.orchestra.run.smithi007.stdout:2753 read 0x75adabf thru 0x75b99f6 (0xbf38 bytes) 2024-09-12T09:30:55.547 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:55.544+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:55.565 INFO:teuthology.orchestra.run.smithi007.stdout:2755 trunc from 0xde7bc74 to 0x4cdbc5a 2024-09-12T09:30:55.576 INFO:teuthology.orchestra.run.smithi007.stdout:2758 read 0xf6531b thru 0xf6cc27 (0x790d bytes) 2024-09-12T09:30:55.576 INFO:teuthology.orchestra.run.smithi007.stdout:2762 read 0x32e8d90 thru 0x32ea042 (0x12b3 bytes) 2024-09-12T09:30:55.576 INFO:teuthology.orchestra.run.smithi007.stdout:2765 punch from 0xf73e27 to 0xf81a6c, (0xdc45 bytes) 2024-09-12T09:30:55.589 INFO:teuthology.orchestra.run.smithi007.stdout:2767 trunc from 0x4cdbc5a to 0xa9e5b28 2024-09-12T09:30:55.599 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.599 INFO:teuthology.orchestra.run.smithi007.stdout:2769 write 0x9c0ba08 thru 0x9c0e06c (0x2665 bytes) 2024-09-12T09:30:55.607 INFO:teuthology.orchestra.run.smithi007.stdout:2770 read 0x2125757 thru 0x212dafc (0x83a6 bytes) 2024-09-12T09:30:55.620 INFO:teuthology.orchestra.run.smithi007.stdout:2771 write 0xb09fec8 thru 0xb0ab24f (0xb388 bytes) 2024-09-12T09:30:55.631 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.631 INFO:teuthology.orchestra.run.smithi007.stdout:2773 read 0x17497 thru 0x26e22 (0xf98c bytes) 2024-09-12T09:30:55.634 INFO:teuthology.orchestra.run.smithi007.stdout:2774 read 0xa813eae thru 0xa8202b7 (0xc40a bytes) 2024-09-12T09:30:55.640 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:55.640 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:55.649 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:55.647+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:55.650 INFO:teuthology.orchestra.run.smithi007.stdout:2775 write 0xd9ec9b2 thru 0xd9f18cd (0x4f1c bytes) 2024-09-12T09:30:55.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:30:55.657 INFO:teuthology.orchestra.run.smithi007.stdout:2776 write 0x557329 thru 0x5592fb (0x1fd3 bytes) 2024-09-12T09:30:55.660 INFO:teuthology.orchestra.run.smithi007.stdout:2777 punch from 0xc21457f to 0xc215f4d, (0x19ce bytes) 2024-09-12T09:30:55.660 INFO:teuthology.orchestra.run.smithi007.stdout:2778 writesame 0x5986ac3 thru 0x5991e60 data_size 0x53(0xb39e bytes) 2024-09-12T09:30:55.668 INFO:teuthology.orchestra.run.smithi007.stdout:2779 write 0x31f8fc7 thru 0x3202ef1 (0x9f2b bytes) 2024-09-12T09:30:55.673 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.673 INFO:teuthology.orchestra.run.smithi007.stdout:2781 writesame 0x892b821 thru 0x893a27b data_size 0x47(0xea5b bytes) 2024-09-12T09:30:55.680 INFO:teuthology.orchestra.run.smithi007.stdout:2784 write 0xb891872 thru 0xb8966cc (0x4e5b bytes) 2024-09-12T09:30:55.683 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:2785 read 0x8b1427d thru 0x8b1d4f9 (0x927d bytes) 2024-09-12T09:30:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:2786 read 0x78b0400 thru 0x78b37eb (0x33ec bytes) 2024-09-12T09:30:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:2788 writesame 0x82ae41d thru 0x82bbd96 data_size 0x1b(0xd97a bytes) 2024-09-12T09:30:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:2789 read 0x121b258 thru 0x1226788 (0xb531 bytes) 2024-09-12T09:30:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:2790 write 0xb48ba93 thru 0xb498d8f (0xd2fd bytes) 2024-09-12T09:30:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:2791 writesame 0x9a9656d thru 0x9a9fc85 data_size 0x1(0x9719 bytes) 2024-09-12T09:30:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:55.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:2792 writesame 0x42e658e thru 0x42ef577 data_size 0x6d(0x8fea bytes) 2024-09-12T09:30:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:2796 read 0xc360aee thru 0xc3636eb (0x2bfe bytes) 2024-09-12T09:30:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:2797 read 0x728cfdd thru 0x728e8d6 (0x18fa bytes) 2024-09-12T09:30:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:55.740 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:2798 trunc from 0xd9f18ce to 0x6d2cf26 2024-09-12T09:30:55.749 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:55.747+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:55.762 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:2800 read 0x3e1d62b thru 0x3e207dc (0x31b2 bytes) 2024-09-12T09:30:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:2801 write 0x5e047e1 thru 0x5e0c17a (0x799a bytes) 2024-09-12T09:30:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:55.782 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:2802 write 0xb83ef21 thru 0xb8486f5 (0x97d5 bytes) 2024-09-12T09:30:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:2804 write 0xd5c28ca thru 0xd5d1311 (0xea48 bytes) 2024-09-12T09:30:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:2805 read 0x1e8e253 thru 0x1e909a4 (0x2752 bytes) 2024-09-12T09:30:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:2808 writesame 0xc1bb913 thru 0xc1c3bba data_size 0x2(0x82a8 bytes) 2024-09-12T09:30:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:2809 read 0x9583a7a thru 0x9586270 (0x27f7 bytes) 2024-09-12T09:30:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:2811 read 0x7723ad7 thru 0x7727d73 (0x429d bytes) 2024-09-12T09:30:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:2812 read 0xc8480bd thru 0xc85265d (0xa5a1 bytes) 2024-09-12T09:30:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:2813 write 0x56c2fa8 thru 0x56cf80f (0xc868 bytes) 2024-09-12T09:30:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:55.826 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:2815 writesame 0x8100a61 thru 0x8102521 data_size 0x3(0x1ac1 bytes) 2024-09-12T09:30:55.834 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:55.834 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:30:55.834 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:30:55.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:30:55.834 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:2816 write 0xad4b8a thru 0xadb4dd (0x6954 bytes) 2024-09-12T09:30:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:2817 trunc from 0xd5d1312 to 0xa0c8734 2024-09-12T09:30:55.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:30:55.850 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:55.847+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:55.862 INFO:teuthology.orchestra.run.smithi007.stdout:2818 read 0x8f1c201 thru 0x8f2a295 (0xe095 bytes) 2024-09-12T09:30:55.862 INFO:teuthology.orchestra.run.smithi007.stdout:2822 writesame 0x84f889c thru 0x850126b data_size 0x78(0x89d0 bytes) 2024-09-12T09:30:55.879 INFO:teuthology.orchestra.run.smithi007.stdout:2824 trunc from 0xa0c8734 to 0x26d5d1b 2024-09-12T09:30:55.897 INFO:teuthology.orchestra.run.smithi007.stdout:2825 read 0x1d6c9f4 thru 0x1d74b75 (0x8182 bytes) 2024-09-12T09:30:55.915 INFO:teuthology.orchestra.run.smithi007.stdout:2826 write 0x8bc4e87 thru 0x8bce454 (0x95ce bytes) 2024-09-12T09:30:55.919 INFO:teuthology.orchestra.run.smithi007.stdout:2827 writesame 0x6301a39 thru 0x630ea5e data_size 0x6(0xd026 bytes) 2024-09-12T09:30:55.927 INFO:teuthology.orchestra.run.smithi007.stdout:2828 write 0x73a6371 thru 0x73b1e4a (0xbada bytes) 2024-09-12T09:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:2830 punch from 0x27b57fc to 0x27c0cd1, (0xb4d5 bytes) 2024-09-12T09:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:2834 trunc from 0x8bce455 to 0x9b6b7a3 2024-09-12T09:30:55.943 INFO:teuthology.orchestra.run.smithi007.stdout:2836 read 0x492b349 thru 0x49338e1 (0x8599 bytes) 2024-09-12T09:30:55.950 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:55.948+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:55.950 INFO:teuthology.orchestra.run.smithi007.stdout:2838 writesame 0xa27339d thru 0xa281dec data_size 0x17(0xea50 bytes) 2024-09-12T09:30:55.963 INFO:teuthology.orchestra.run.smithi007.stdout:2839 write 0xd4e3d76 thru 0xd4ec9dd (0x8c68 bytes) 2024-09-12T09:30:55.970 INFO:teuthology.orchestra.run.smithi007.stdout:2840 write 0x6264489 thru 0x6264cb9 (0x831 bytes) 2024-09-12T09:30:55.975 INFO:teuthology.orchestra.run.smithi007.stdout:2841 write 0x19bf32 thru 0x1a74cd (0xb59c bytes) 2024-09-12T09:30:55.984 INFO:teuthology.orchestra.run.smithi007.stdout:2842 writesame 0x422bb1f thru 0x422d237 data_size 0x3(0x1719 bytes) 2024-09-12T09:30:55.990 INFO:teuthology.orchestra.run.smithi007.stdout:2843 write 0x5aa06bc thru 0x5aab9e3 (0xb328 bytes) 2024-09-12T09:30:55.996 INFO:teuthology.orchestra.run.smithi007.stdout:2844 clone 30 order 23 su 131072 sc 12 2024-09-12T09:30:56.024 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:30:56.024 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:30:56.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:30:56.024 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2479 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14213120~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625718023999994, 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049863431, 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2479 2024-09-12T09:30:56.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397666+0000", 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397668+0000", 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397673+0000", 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397690+0000", 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039000000000001e-05 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397705+0000", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.42e-05 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447242+0000", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537052999999998 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447378+0000", 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000136057 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447530+0000", 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000152075 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2480 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14245888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625661437999995, 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050109064000000002, 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.029 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2480 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397723+0000", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397725+0000", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7829999999999999e-06 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397740+0000", 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5094e-05 2024-09-12T09:30:56.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397758+0000", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8410000000000002e-05 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397772+0000", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447578+0000", 2024-09-12T09:30:56.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049805800999999997 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447691+0000", 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011285699999999999 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447832+0000", 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014155600000000001 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2481 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14278656~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625593124999995, 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050337451999999998, 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2481 2024-09-12T09:30:56.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397791+0000", 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397792+0000", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.144e-06 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397798+0000", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5600000000000001e-06 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397818+0000", 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9796000000000002e-05 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397831+0000", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3482999999999999e-05 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447874+0000", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042746999999999 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.447984+0000", 2024-09-12T09:30:56.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011031600000000001 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448129+0000", 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000144406 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2482 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14311424~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:56.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625534939000005, 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050576967, 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2482 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:56.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397850+0000", 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397851+0000", 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397856+0000", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2560000000000002e-06 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397873+0000", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7484000000000001e-05 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397888+0000", 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4276e-05 2024-09-12T09:30:56.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448175+0000", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050286823000000001 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448286+0000", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111692 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448426+0000", 2024-09-12T09:30:56.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027899999999999 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2483 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14344192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625474405999995, 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050809638999999997, 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2483 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397910+0000", 2024-09-12T09:30:56.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397913+0000", 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3469999999999999e-06 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397919+0000", 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397936+0000", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7220000000000001e-05 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397951+0000", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4354e-05 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448467+0000", 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05051655 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448580+0000", 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113406 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448720+0000", 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013919900000000001 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2484 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14376960~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625414406000004, 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051182310000000002, 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2484 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:56.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397970+0000", 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397971+0000", 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397977+0000", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.397995+0000", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7813e-05 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398008+0000", 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3370000000000001e-05 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448824+0000", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050816369 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.448976+0000", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015132599999999999 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449152+0000", 2024-09-12T09:30:56.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000176782 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2485 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14409728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625358057, 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051444872000000003, 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2485 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398026+0000", 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398028+0000", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398034+0000", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2199999999999997e-06 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398054+0000", 2024-09-12T09:30:56.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9864000000000002e-05 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398068+0000", 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3730000000000001e-05 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449201+0000", 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051133350000000001 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449327+0000", 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012594900000000001 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449471+0000", 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014442900000000001 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2486 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14442496~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625298747000002, 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051680643999999998, 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2486 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398086+0000", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398087+0000", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398092+0000", 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398110+0000", 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7779999999999999e-05 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398123+0000", 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3563e-05 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449513+0000", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051389907999999998 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449625+0000", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011217699999999999 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449766+0000", 2024-09-12T09:30:56.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014112900000000001 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2487 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14475264~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625243114, 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051926938999999998, 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2487 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398141+0000", 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398143+0000", 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398148+0000", 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9499999999999998e-06 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398165+0000", 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6872999999999999e-05 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398179+0000", 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3366999999999999e-05 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449808+0000", 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051629130000000002 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.449923+0000", 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114867 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450068+0000", 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000145592 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2488 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14508032~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625186730999999, 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052165177, 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2488 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398198+0000", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398199+0000", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.463e-06 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398205+0000", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398222+0000", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7237000000000001e-05 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398235+0000", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3613e-05 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450112+0000", 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051876879000000001 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450223+0000", 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450363+0000", 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014027199999999999 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2489 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14540800~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625130161000001, 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052521228000000003, 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2489 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398254+0000", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398255+0000", 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.153e-06 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398261+0000", 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398279+0000", 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8082999999999999e-05 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398292+0000", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.34e-05 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450449+0000", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052156876999999997 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450597+0000", 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000148145 2024-09-12T09:30:56.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450776+0000", 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000178476 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2490 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14573568~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625073744999995, 2024-09-12T09:30:56.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052792100000000002, 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2490 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398311+0000", 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398312+0000", 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3999999999999999e-06 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398318+0000", 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4399999999999996e-06 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398334+0000", 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6803000000000002e-05 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398348+0000", 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3307e-05 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450839+0000", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052491308 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.450959+0000", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119599 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451103+0000", 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014424300000000001 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2491 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14606336~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.625018252000004, 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053080337999999998, 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2491 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398366+0000", 2024-09-12T09:30:56.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398367+0000", 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1829999999999999e-06 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398373+0000", 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2800000000000003e-06 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398391+0000", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7904e-05 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398405+0000", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451166+0000", 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052761389999999998 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451292+0000", 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012538600000000001 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451447+0000", 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000154969 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2492 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14639104~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.624930238999994, 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0533188, 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2492 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398454+0000", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398456+0000", 2024-09-12T09:30:56.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.587e-06 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398462+0000", 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2129999999999996e-06 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398482+0000", 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0326000000000002e-05 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398497+0000", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4367e-05 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451508+0000", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053011338999999998 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451625+0000", 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117432 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451773+0000", 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014753600000000001 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:2493 3.11 3:8daee3ab:::rbd_data.1191d634a7ff.0000000000000009:5 [read 14671872~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.624865795999995, 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053561298, 2024-09-12T09:30:56.088 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2493 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:56.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398519+0000", 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398520+0000", 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.516e-06 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398526+0000", 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398544+0000", 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7759999999999999e-05 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.398560+0000", 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6203999999999999e-05 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451807+0000", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053247496999999998 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.451929+0000", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012215500000000001 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.452080+0000", 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000150576 2024-09-12T09:30:56.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49738 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15960775~34627] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.778418010000003, 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.11883020499999999, 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49738 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244966+0000", 2024-09-12T09:30:56.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.244974+0000", 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7659999999999998e-06 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282596+0000", 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037621614999999997 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282638+0000", 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2040999999999999e-05 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282657+0000", 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9626000000000001e-05 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.282837+0000", 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017994699999999999 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.283245+0000", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040802600000000003 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.284947+0000", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001701473 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363216+0000", 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.078268584000000002 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363562+0000", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00034645999999999999 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363704+0000", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0001418 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.363797+0000", 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2867000000000004e-05 2024-09-12T09:30:56.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49866 3.12 e50/41 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head v 50'111, mlcod=49'108)", 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.595001824999997, 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.081307475000000004, 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.428383+0000", 2024-09-12T09:30:56.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.99e-07 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463171+0000", 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034787797000000002 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463221+0000", 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9756999999999998e-05 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463240+0000", 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9015999999999999e-05 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463370+0000", 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013023000000000001 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.463416+0000", 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5991000000000001e-05 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509590+0000", 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046173829999999999 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.509690+0000", 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010025499999999999 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:49995 3.15 3:ac9ea161:::rbd_data.119172280d2a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.478935722000003, 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.090543324999999994, 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 49995 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544449+0000", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.544452+0000", 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-12T09:30:56.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574420+0000", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029968136999999999 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574473+0000", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2784000000000001e-05 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574500+0000", 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7035000000000001e-05 2024-09-12T09:30:56.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574621+0000", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012105800000000001 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.574797+0000", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017587899999999999 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.575718+0000", 2024-09-12T09:30:56.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092105300000000002 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.633854+0000", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.058136515999999999 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634780+0000", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00092573999999999998 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634896+0000", 2024-09-12T09:30:56.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116554 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.634992+0000", 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5599000000000004e-05 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:50126 3.e 3:71c6ffaa:::rbd_data.119172280d2a.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10124644~56950] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.365905863999998, 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.084419143000000002, 2024-09-12T09:30:56.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 50126 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657479+0000", 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.657481+0000", 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680415+0000", 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022933843999999998 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680456+0000", 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0627999999999998e-05 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680472+0000", 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5809000000000001e-05 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680587+0000", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115598 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.680870+0000", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028296099999999998 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.682154+0000", 2024-09-12T09:30:56.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283913 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.717384+0000", 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035229531000000001 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741764+0000", 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024380807000000001 2024-09-12T09:30:56.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741834+0000", 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9276999999999994e-05 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.741898+0000", 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3979999999999994e-05 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:142432 3.7 3:e7a38470:::rbd_data.1191f803704b.0000000000000252:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:52.649563+0000", 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.3738213259999998, 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054324909999999997, 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 142432 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.116 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649563+0000", 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649563+0000", 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:30:56.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649566+0000", 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6800000000000002e-06 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649578+0000", 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1708999999999999e-05 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649609+0000", 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1377000000000001e-05 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649625+0000", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5656000000000001e-05 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649741+0000", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116475 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.649959+0000", 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021782299999999999 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.660807+0000", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010848327 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.703530+0000", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042722669999999997 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.703660+0000", 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012977 2024-09-12T09:30:56.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.703761+0000", 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101069 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:52.703888+0000", 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012735399999999999 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:30:56.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:30:56.122 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:30:56.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:30:56.123 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:56.120+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:56.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:30:56.213 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone29 from 0xd4ec9de (overlap 0xaa7c92) to 0x664abe 2024-09-12T09:30:56.225 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:56.223+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:56.269 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:56.269 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:56.269 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:56.269 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:56.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:30:56.340 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:56.340+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:56.363 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-12T09:30:56.411 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:56.411 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:56.427 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:56.424+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:56.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:30:56.528 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:56.528+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:30:56.602 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.462556001000003, 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:30:56.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:30:56.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:30:56.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:30:56.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:56.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361220459000002, 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:56.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:30:56.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.618 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361195006999999, 2024-09-12T09:30:56.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:30:56.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:30:56.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:30:56.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:30:56.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361170155000003, 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:30:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361144817000003, 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:30:56.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361113787000001, 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.638 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:30:56.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:30:56.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:30:56.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361088625999997, 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:56.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361062726, 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:30:56.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:30:56.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:56.655 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361039120000001, 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:56.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:30:56.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.361014607999998, 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:30:56.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360990170000001, 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:30:56.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:30:56.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360965583000002, 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.670 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:30:56.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360941566999998, 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:30:56.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360913725000003, 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:30:56.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360888879999997, 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:30:56.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:30:56.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360864153999998, 2024-09-12T09:30:56.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:30:56.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.360839657, 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:30:56.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.355033786, 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:30:56.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.210081821999999, 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:30:56.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:30:56.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:30:56.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.392585023999999, 2024-09-12T09:30:56.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:56.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:56.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:56.714 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:56.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:30:56.716 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:56.712+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:56.816 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:56.815+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:56.817 INFO:teuthology.orchestra.run.smithi007.stdout:2846 read 0x778ddb3 thru 0x7792bb8 (0x4e06 bytes) 2024-09-12T09:30:56.818 INFO:teuthology.orchestra.run.smithi007.stdout:2848 read 0x150dff9 thru 0x151996f (0xb977 bytes) 2024-09-12T09:30:56.819 INFO:teuthology.orchestra.run.smithi007.stdout:2849 writesame 0x2138d3a thru 0x2146b32 data_size 0x5(0xddf9 bytes) 2024-09-12T09:30:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:2850 trunc from 0xd4ec9de to 0x3b20afc 2024-09-12T09:30:56.904 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:56.904 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:56.904 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:30:56.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:56.917 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:56.915+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:56.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:30:56.953 INFO:teuthology.orchestra.run.smithi007.stdout:2851 write 0xb26c6b1 thru 0xb27543a (0x8d8a bytes) 2024-09-12T09:30:56.959 INFO:teuthology.orchestra.run.smithi007.stdout:2852 writesame 0x54515b2 thru 0x54567ec data_size 0x3(0x523b bytes) 2024-09-12T09:30:56.963 INFO:teuthology.orchestra.run.smithi007.stdout:2853 write 0x59d9f8e thru 0x59e4e92 (0xaf05 bytes) 2024-09-12T09:30:56.966 INFO:teuthology.orchestra.run.smithi007.stdout:2854 writesame 0x6c1f91f thru 0x6c28807 data_size 0x2d(0x8ee9 bytes) 2024-09-12T09:30:56.974 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:56.984 INFO:teuthology.orchestra.run.smithi007.stdout:2857 write 0xb8b349e thru 0xb8c08d6 (0xd439 bytes) 2024-09-12T09:30:56.992 INFO:teuthology.orchestra.run.smithi007.stdout:2858 read 0x81a2487 thru 0x81aa570 (0x80ea bytes) 2024-09-12T09:30:57.006 INFO:teuthology.orchestra.run.smithi007.stdout:2860 write 0xdcdf7af thru 0xdce0d3c (0x158e bytes) 2024-09-12T09:30:57.014 INFO:teuthology.orchestra.run.smithi007.stdout:2862 read 0x55a018a thru 0x55a96fd (0x9574 bytes) 2024-09-12T09:30:57.015 INFO:teuthology.orchestra.run.smithi007.stdout:2863 writesame 0x7ca331a thru 0x7ca4eb3 data_size 0x1(0x1b9a bytes) 2024-09-12T09:30:57.017 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:57.016+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:57.024 INFO:teuthology.orchestra.run.smithi007.stdout:2864 write 0x822dfad thru 0x8239635 (0xb689 bytes) 2024-09-12T09:30:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:2865 read 0x8a2ef5c thru 0x8a3aa50 (0xbaf5 bytes) 2024-09-12T09:30:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:2866 read 0x26d3112 thru 0x26d8b08 (0x59f7 bytes) 2024-09-12T09:30:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:2867 read 0x298bb10 thru 0x298fae4 (0x3fd5 bytes) 2024-09-12T09:30:57.045 INFO:teuthology.orchestra.run.smithi007.stdout:2868 writesame 0xe0bdf21 thru 0xe0c3946 data_size 0x2(0x5a26 bytes) 2024-09-12T09:30:57.045 INFO:teuthology.orchestra.run.smithi007.stdout:2871 read 0x597438 thru 0x598650 (0x1219 bytes) 2024-09-12T09:30:57.046 INFO:teuthology.orchestra.run.smithi007.stdout:2873 punch from 0x2db4c88 to 0x2db9a52, (0x4dca bytes) 2024-09-12T09:30:57.046 INFO:teuthology.orchestra.run.smithi007.stdout:2874 punch from 0x28981fa to 0x28a5bc9, (0xd9cf bytes) 2024-09-12T09:30:57.046 INFO:teuthology.orchestra.run.smithi007.stdout:2875 read 0x64311bd thru 0x6434dd1 (0x3c15 bytes) 2024-09-12T09:30:57.046 INFO:teuthology.orchestra.run.smithi007.stdout:2877 punch from 0x32819c to 0x3352e6, (0xd14a bytes) 2024-09-12T09:30:57.047 INFO:teuthology.orchestra.run.smithi007.stdout:2878 write 0x712c0af thru 0x712f521 (0x3473 bytes) 2024-09-12T09:30:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:57.054 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:57.054 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:57.054 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:57.068 INFO:teuthology.orchestra.run.smithi007.stdout:2881 trunc from 0xe0c3947 to 0x62905ba 2024-09-12T09:30:57.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:30:57.086 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:30:57.105 INFO:teuthology.orchestra.run.smithi007.stdout:2882 write 0x4bd0089 thru 0x4bdaa7e (0xa9f6 bytes) 2024-09-12T09:30:57.116 INFO:teuthology.orchestra.run.smithi007.stdout:2883 trunc from 0x62905ba to 0x142e5ac 2024-09-12T09:30:57.118 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:57.116+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:57.145 INFO:teuthology.orchestra.run.smithi007.stdout:2884 read 0x1200405 thru 0x120649a (0x6096 bytes) 2024-09-12T09:30:57.168 INFO:teuthology.orchestra.run.smithi007.stdout:2885 write 0xada07a6 thru 0xada6e64 (0x66bf bytes) 2024-09-12T09:30:57.174 INFO:teuthology.orchestra.run.smithi007.stdout:2887 writesame 0x89a906a thru 0x89b7424 data_size 0x1(0xe3bb bytes) 2024-09-12T09:30:57.216 INFO:teuthology.orchestra.run.smithi007.stdout:2888 write 0xa12f47b thru 0xa137a66 (0x85ec bytes) 2024-09-12T09:30:57.219 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:57.216+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:57.225 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:57.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.024145421, 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:30:57.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:30:57.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.009174692000002, 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.692327687999999, 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:30:57.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.692281329, 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:57.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.692174109, 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:57.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:30:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.692080617999999, 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:30:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:30:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.692014104999998, 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:30:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:30:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691953296000001, 2024-09-12T09:30:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:30:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691894004000002, 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:30:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:30:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691833495000001, 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:30:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691755440000001, 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:30:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:30:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:30:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691670065, 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:30:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691601746, 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:30:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691509160999999, 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:30:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691453251999999, 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:30:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691420929, 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:30:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:30:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691387335999998, 2024-09-12T09:30:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:30:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691336394, 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:30:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691238239, 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:30:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:30:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.691181145000002, 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:30:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.321 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:57.322 INFO:teuthology.orchestra.run.smithi007.stdout:2890 flatten 2024-09-12T09:30:57.322 INFO:teuthology.orchestra.run.smithi007.stdout:2893 punch from 0x1b840ce to 0x1b8c470, (0x83a2 bytes) 2024-09-12T09:30:57.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:30:57.324 INFO:teuthology.orchestra.run.smithi007.stdout:2895 trunc from 0xada6e65 to 0xa1a9073 2024-09-12T09:30:57.324 INFO:teuthology.orchestra.run.smithi007.stdout:2896 read 0x42ed63c thru 0x42f4fd3 (0x7998 bytes) 2024-09-12T09:30:57.324 INFO:teuthology.orchestra.run.smithi007.stdout:2897 read 0xa122ff4 thru 0xa12c46b (0x9478 bytes) 2024-09-12T09:30:57.324 INFO:teuthology.orchestra.run.smithi007.stdout:2898 write 0xb565f83 thru 0xb566bc5 (0xc43 bytes) 2024-09-12T09:30:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:2899 write 0x2a14402 thru 0x2a23add (0xf6dc bytes) 2024-09-12T09:30:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:2900 writesame 0x81cd07a thru 0x81d7168 data_size 0x1f(0xa0ef bytes) 2024-09-12T09:30:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:2902 trunc from 0xb566bc6 to 0x7decf03 2024-09-12T09:30:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:2903 read 0x216d998 thru 0x2174998 (0x7001 bytes) 2024-09-12T09:30:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:2904 trunc from 0x7decf03 to 0x6d23bbd 2024-09-12T09:30:57.365 INFO:teuthology.orchestra.run.smithi007.stdout:2906 write 0xaac16f7 thru 0xaacecad (0xd5b7 bytes) 2024-09-12T09:30:57.366 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:57.364+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:57.380 INFO:teuthology.orchestra.run.smithi007.stdout:2908 trunc from 0xaacecae to 0x80f0eb4 2024-09-12T09:30:57.394 INFO:teuthology.orchestra.run.smithi007.stdout:2909 write 0x6c1fdd5 thru 0x6c2296a (0x2b96 bytes) 2024-09-12T09:30:57.403 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.416 INFO:teuthology.orchestra.run.smithi007.stdout:2912 write 0x96382d0 thru 0x963ee40 (0x6b71 bytes) 2024-09-12T09:30:57.419 INFO:teuthology.orchestra.run.smithi007.stdout:2914 writesame 0x677ba55 thru 0x677dafe data_size 0x25(0x20aa bytes) 2024-09-12T09:30:57.422 INFO:teuthology.orchestra.run.smithi007.stdout:2915 write 0x4312d8d thru 0x431a831 (0x7aa5 bytes) 2024-09-12T09:30:57.425 INFO:teuthology.orchestra.run.smithi007.stdout:2917 read 0x82aca4d thru 0x82b725a (0xa80e bytes) 2024-09-12T09:30:57.426 INFO:teuthology.orchestra.run.smithi007.stdout:2918 punch from 0x6dc2dbd to 0x6dc519c, (0x23df bytes) 2024-09-12T09:30:57.427 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:57.424+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:57.427 INFO:teuthology.orchestra.run.smithi007.stdout:2920 writesame 0x5314592 thru 0x5321488 data_size 0x57(0xcef7 bytes) 2024-09-12T09:30:57.430 INFO:teuthology.orchestra.run.smithi007.stdout:2921 read 0x564c1d0 thru 0x564c983 (0x7b4 bytes) 2024-09-12T09:30:57.445 INFO:teuthology.orchestra.run.smithi007.stdout:2924 write 0xc1eca39 thru 0xc1f252a (0x5af2 bytes) 2024-09-12T09:30:57.452 INFO:teuthology.orchestra.run.smithi007.stdout:2928 punch from 0x238007e to 0x2389539, (0x94bb bytes) 2024-09-12T09:30:57.452 INFO:teuthology.orchestra.run.smithi007.stdout:2929 read 0x3d5af95 thru 0x3d69857 (0xe8c3 bytes) 2024-09-12T09:30:57.453 INFO:teuthology.orchestra.run.smithi007.stdout:2930 read 0x804ae2e thru 0x8053d58 (0x8f2b bytes) 2024-09-12T09:30:57.453 INFO:teuthology.orchestra.run.smithi007.stdout:2931 writesame 0x71c77ce thru 0x71d3f09 data_size 0x52(0xc73c bytes) 2024-09-12T09:30:57.458 INFO:teuthology.orchestra.run.smithi007.stdout:2932 read 0xa3b958a thru 0xa3c63a7 (0xce1e bytes) 2024-09-12T09:30:57.459 INFO:teuthology.orchestra.run.smithi007.stdout:2937 punch from 0x9db13f1 to 0x9dbcbe5, (0xb7f4 bytes) 2024-09-12T09:30:57.459 INFO:teuthology.orchestra.run.smithi007.stdout:2940 read 0x8c99bde thru 0x8c9e6c5 (0x4ae8 bytes) 2024-09-12T09:30:57.459 INFO:teuthology.orchestra.run.smithi007.stdout:2941 writesame 0x4029888 thru 0x4036bd7 data_size 0x54(0xd350 bytes) 2024-09-12T09:30:57.463 INFO:teuthology.orchestra.run.smithi007.stdout:2942 read 0x8634833 thru 0x86369c9 (0x2197 bytes) 2024-09-12T09:30:57.472 INFO:teuthology.orchestra.run.smithi007.stdout:2944 trunc from 0xc1f252b to 0x73a0d75 2024-09-12T09:30:57.517 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:145890 3.1a e93/83)", 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:57.517469+0000", 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00010286299999999999, 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014099300000000001, 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517469+0000", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517469+0000", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517472+0000", 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-06 2024-09-12T09:30:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517489+0000", 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.628e-05 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517526+0000", 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7086999999999999e-05 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:57.517543+0000", 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7e-05 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-12T09:30:57.521 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:57.527 INFO:teuthology.orchestra.run.smithi007.stdout:2945 writesame 0xe224393 thru 0xe225139 data_size 0x5(0xda7 bytes) 2024-09-12T09:30:57.527 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.527 INFO:teuthology.orchestra.run.smithi007.stdout:2949 writesame 0x119f696 thru 0x11a724d data_size 0x4a(0x7bb8 bytes) 2024-09-12T09:30:57.528 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:57.524+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:57.532 INFO:teuthology.orchestra.run.smithi007.stdout:2950 write 0xdce7829 thru 0xdcf43e4 (0xcbbc bytes) 2024-09-12T09:30:57.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:30:57.540 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.540 INFO:teuthology.orchestra.run.smithi007.stdout:2952 punch from 0x9318389 to 0x9325640, (0xd2b7 bytes) 2024-09-12T09:30:57.552 INFO:teuthology.orchestra.run.smithi007.stdout:2953 trunc from 0xe22513a to 0x8aed4b0 2024-09-12T09:30:57.575 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.581 INFO:teuthology.orchestra.run.smithi007.stdout:2955 trunc from 0x8aed4b0 to 0x5174d0e 2024-09-12T09:30:57.606 INFO:teuthology.orchestra.run.smithi007.stdout:2957 punch from 0xb4969f to 0xb4c2be, (0x2c1f bytes) 2024-09-12T09:30:57.628 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:57.627+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:57.630 INFO:teuthology.orchestra.run.smithi007.stdout:2958 write 0xed23f09 thru 0xed283e7 (0x44df bytes) 2024-09-12T09:30:57.636 INFO:teuthology.orchestra.run.smithi007.stdout:2959 write 0xaa8dd37 thru 0xaa9afb5 (0xd27f bytes) 2024-09-12T09:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout:2960 punch from 0xc464c99 to 0xc46efb9, (0xa320 bytes) 2024-09-12T09:30:57.642 INFO:teuthology.orchestra.run.smithi007.stdout:2961 write 0x27abea3 thru 0x27b1236 (0x5394 bytes) 2024-09-12T09:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout:2962 read 0x8a6f066 thru 0x8a78127 (0x90c2 bytes) 2024-09-12T09:30:57.645 INFO:teuthology.orchestra.run.smithi007.stdout:2963 read 0x10d2d9e thru 0x10e0aa7 (0xdd0a bytes) 2024-09-12T09:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout:2966 read 0xbcb0e79 thru 0xbcb7608 (0x6790 bytes) 2024-09-12T09:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.646 INFO:teuthology.orchestra.run.smithi007.stdout:2972 read 0x50588ef thru 0x505b8f4 (0x3006 bytes) 2024-09-12T09:30:57.647 INFO:teuthology.orchestra.run.smithi007.stdout:2974 write 0x11f90 thru 0x1ec50 (0xccc1 bytes) 2024-09-12T09:30:57.650 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.651 INFO:teuthology.orchestra.run.smithi007.stdout:2976 writesame 0x212ce02 thru 0x212eaeb data_size 0x1(0x1cea bytes) 2024-09-12T09:30:57.663 INFO:teuthology.orchestra.run.smithi007.stdout:2977 trunc from 0xed283e8 to 0xdacadea 2024-09-12T09:30:57.677 INFO:teuthology.orchestra.run.smithi007.stdout:2978 write 0xc94b3ab thru 0xc9583ba (0xd010 bytes) 2024-09-12T09:30:57.680 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:30:57.681 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout:2980 read 0xc391414 thru 0xc39c0a5 (0xac92 bytes) 2024-09-12T09:30:57.684 INFO:teuthology.orchestra.run.smithi007.stdout:2981 punch from 0x240c751 to 0x241a318, (0xdbc7 bytes) 2024-09-12T09:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout:2982 punch from 0x96ca47a to 0x96cbf10, (0x1a96 bytes) 2024-09-12T09:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout:2983 punch from 0x24d088b to 0x24d1b9b, (0x1310 bytes) 2024-09-12T09:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout:2985 read 0x2940e0b thru 0x29412c6 (0x4bc bytes) 2024-09-12T09:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout:2986 read 0x829a16c thru 0x82a41eb (0xa080 bytes) 2024-09-12T09:30:57.685 INFO:teuthology.orchestra.run.smithi007.stdout:2987 write 0x9f5b4d9 thru 0x9f5faa6 (0x45ce bytes) 2024-09-12T09:30:57.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:30:57.709 INFO:teuthology.orchestra.run.smithi007.stdout:2989 trunc from 0xdacadea to 0x47060ce 2024-09-12T09:30:57.729 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:57.727+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:57.754 INFO:teuthology.orchestra.run.smithi007.stdout:2990 writesame 0xbc0438c thru 0xbc0c022 data_size 0x1(0x7c97 bytes) 2024-09-12T09:30:57.754 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:57.765 INFO:teuthology.orchestra.run.smithi007.stdout:2992 writesame 0xd3e596b thru 0xd3eda38 data_size 0x1(0x80ce bytes) 2024-09-12T09:30:57.766 INFO:teuthology.orchestra.run.smithi007.stdout:2993 read 0xc7132da thru 0xc71de66 (0xab8d bytes) 2024-09-12T09:30:57.766 INFO:teuthology.orchestra.run.smithi007.stdout:2995 read 0xbbbf52a thru 0xbbc6b85 (0x765c bytes) 2024-09-12T09:30:57.766 INFO:teuthology.orchestra.run.smithi007.stdout:2996 write 0x3eb2cff thru 0x3eb41dc (0x14de bytes) 2024-09-12T09:30:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:2997 writesame 0x71651f1 thru 0x7172794 data_size 0x79(0xd5a4 bytes) 2024-09-12T09:30:57.775 INFO:teuthology.orchestra.run.smithi007.stdout:2998 trunc from 0xd3eda39 to 0xd4ced30 2024-09-12T09:30:57.801 INFO:teuthology.orchestra.run.smithi007.stdout:2999 trunc from 0xd4ced30 to 0x3595f1c 2024-09-12T09:30:57.829 INFO:teuthology.orchestra.run.smithi007.stdout:3000 read 0xd65dd7 thru 0xd6fc1c (0x9e46 bytes) 2024-09-12T09:30:57.830 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:57.827+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:57.843 INFO:teuthology.orchestra.run.smithi007.stdout:3001 writesame 0x38b1a97 thru 0x38b5980 data_size 0x1(0x3eea bytes) 2024-09-12T09:30:57.846 INFO:teuthology.orchestra.run.smithi007.stdout:3002 read 0xdabb6e thru 0xdbae1e (0xf2b1 bytes) 2024-09-12T09:30:57.847 INFO:teuthology.orchestra.run.smithi007.stdout:3003 write 0x2db87f2 thru 0x2dc206c (0x987b bytes) 2024-09-12T09:30:57.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:30:57.853 INFO:teuthology.orchestra.run.smithi007.stdout:3004 trunc from 0x38b5981 to 0x24e3f76 2024-09-12T09:30:57.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:30:57.855 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:30:57.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:30:57.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:30:57.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.552386671999997, 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:30:57.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.860 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.369967754000001, 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:30:57.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:30:57.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:30:57.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.977181240999997, 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:30:57.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:30:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.629651353, 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:30:57.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:30:57.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:30:57.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:30:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.627197443, 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:30:57.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.625087555999997, 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:30:57.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:30:57.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141909 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 30275822~43981] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.604702+0000", 2024-09-12T09:30:57.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2470989609999998, 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091620929999999996, 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141909 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.604702+0000", 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.604702+0000", 2024-09-12T09:30:57.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.604705+0000", 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.326e-06 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.606399+0000", 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016941910000000001 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.606437+0000", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7342999999999998e-05 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.606452+0000", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5879000000000002e-05 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.606563+0000", 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110466 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.606815+0000", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025203299999999998 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.607291+0000", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000476009 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.613065+0000", 2024-09-12T09:30:57.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057744190000000003 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.613668+0000", 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060218399999999997 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.613783+0000", 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115758 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.613864+0000", 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0484999999999999e-05 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1293409109999999, 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:30:57.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:30:57.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:30:57.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:30:57.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0989326249999998, 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141952 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.780009+0000", 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0717913729999999, 2024-09-12T09:30:57.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071009660000000002, 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141952 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780009+0000", 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780009+0000", 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780012+0000", 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9730000000000002e-06 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780029+0000", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6923999999999999e-05 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780063+0000", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4099e-05 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780089+0000", 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-12T09:30:57.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780271+0000", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018183199999999999 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.780406+0000", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013535500000000001 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.781501+0000", 2024-09-12T09:30:57.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001094921 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.786924+0000", 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054233340000000001 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.786979+0000", 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4639000000000003e-05 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.787053+0000", 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4176000000000003e-05 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.787110+0000", 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7033000000000001e-05 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8779680660000002, 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.913 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:30:57.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:30:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:142537 3.0 e89/83 ondisk, result = 0)", 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.982497+0000", 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.8693035680000003, 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079118939999999992, 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982497+0000", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982497+0000", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982498+0000", 2024-09-12T09:30:57.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0419999999999999e-06 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982503+0000", 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7550000000000002e-06 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982521+0000", 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7844999999999999e-05 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.982533+0000", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2053e-05 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990366+0000", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078336929999999992 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990382+0000", 2024-09-12T09:30:57.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5679999999999999e-05 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990409+0000", 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6826e-05 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:143817 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:55.784822+0000", 2024-09-12T09:30:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0669786179999998, 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075672409999999997, 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 143817 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784822+0000", 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784822+0000", 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784822+0000", 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.67e-07 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784831+0000", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784835+0000", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8740000000000003e-06 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.784840+0000", 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.301e-06 2024-09-12T09:30:57.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.786783+0000", 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019431330000000001 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.786835+0000", 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1759999999999999e-05 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.787039+0000", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000204267 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.787661+0000", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062167799999999997 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.792311+0000", 2024-09-12T09:30:57.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0046506789999999996 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.792366+0000", 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4985000000000003e-05 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.792389+0000", 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2988000000000001e-05 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:143833 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:55.806713+0000", 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0450873679999999, 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071540179999999998, 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 143833 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806713+0000", 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806713+0000", 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806714+0000", 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8999999999999998e-07 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806716+0000", 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806728+0000", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2493999999999999e-05 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806732+0000", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9620000000000004e-06 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808068+0000", 2024-09-12T09:30:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001336032 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808120+0000", 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1588000000000001e-05 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808355+0000", 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0002352 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.809995+0000", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016403209999999999 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813806+0000", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038110850000000001 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813842+0000", 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5505e-05 2024-09-12T09:30:57.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813867+0000", 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5491e-05 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:143834 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:55.806738+0000", 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.0450625310000001, 2024-09-12T09:30:57.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072208990000000002, 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 143834 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806738+0000", 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806738+0000", 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806738+0000", 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806742+0000", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1360000000000001e-06 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806746+0000", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.98e-06 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806749+0000", 2024-09-12T09:30:57.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.552e-06 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808707+0000", 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019581170000000001 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808759+0000", 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1952e-05 2024-09-12T09:30:57.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.808965+0000", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020606199999999999 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.809999+0000", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001033494 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813908+0000", 2024-09-12T09:30:57.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003909053 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813937+0000", 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8937000000000001e-05 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813959+0000", 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249e-05 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:143835 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:55.806753+0000", 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.045047185, 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073022199999999999, 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 143835 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806753+0000", 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806753+0000", 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806754+0000", 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806755+0000", 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7150000000000001e-06 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806759+0000", 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3790000000000002e-06 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.806762+0000", 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.678e-06 2024-09-12T09:30:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.809409+0000", 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026478539999999998 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.809463+0000", 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3226000000000002e-05 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.809679+0000", 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021646999999999999 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.810296+0000", 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061671399999999998 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.813990+0000", 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036943800000000001 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.814025+0000", 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4552999999999999e-05 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:55.814056+0000", 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0975000000000001e-05 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:144151 3.4 3:268a4316:::rbd_data.1191b7249abe.000000000000005b:head [delete] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:56.236304+0000", 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.615496568, 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091761629999999993, 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 144151 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236304+0000", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236304+0000", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236306+0000", 2024-09-12T09:30:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8929999999999999e-06 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236313+0000", 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236330+0000", 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6742999999999999e-05 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236337+0000", 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5530000000000004e-06 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236403+0000", 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6262999999999999e-05 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236495+0000", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1181000000000007e-05 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.236709+0000", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021453299999999999 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.238849+0000", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021399890000000001 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.245372+0000", 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065225689999999998 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.245433+0000", 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1700999999999998e-05 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.245480+0000", 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6878000000000001e-05 2024-09-12T09:30:57.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:144182 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:56.258637+0000", 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.5931640410000001, 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084637670000000005, 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 144182 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258637+0000", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258637+0000", 2024-09-12T09:30:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258637+0000", 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3900000000000005e-07 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258640+0000", 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1540000000000002e-06 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258647+0000", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258654+0000", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6699999999999997e-06 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.261284+0000", 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026301079999999999 2024-09-12T09:30:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.261372+0000", 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7884000000000001e-05 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.261731+0000", 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035915900000000001 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.264137+0000", 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024057570000000001 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.266999+0000", 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028624029999999999 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.267061+0000", 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1796000000000003e-05 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.267100+0000", 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9289999999999998e-05 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:144183 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:56.258683+0000", 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.593117924, 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0085943570000000004, 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 144183 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258683+0000", 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258683+0000", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258684+0000", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9999999999999996e-07 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258688+0000", 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1080000000000003e-06 2024-09-12T09:30:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258699+0000", 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258705+0000", 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2559999999999999e-06 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.262272+0000", 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035676689999999999 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.262367+0000", 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4322999999999998e-05 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.262769+0000", 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040192599999999999 2024-09-12T09:30:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.264320+0000", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015514719999999999 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.267156+0000", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028361889999999998 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.267231+0000", 2024-09-12T09:30:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4884999999999999e-05 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.267277+0000", 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859000000000003e-05 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:144184 3.4 3:275ee920:::rbd_object_map.1191b7249abe:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:56.258725+0000", 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.5930754469999999, 2024-09-12T09:30:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012770775999999999, 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 144184 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:30:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258725+0000", 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258725+0000", 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258726+0000", 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258728+0000", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.695e-06 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258741+0000", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2407e-05 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.258745+0000", 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2969999999999997e-06 2024-09-12T09:30:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.263313+0000", 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045675239999999999 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.263399+0000", 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6524e-05 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:30:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.263640+0000", 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024141200000000001 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.264330+0000", 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068970399999999997 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.271404+0000", 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070743070000000002 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.271446+0000", 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1921000000000001e-05 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:56.271496+0000", 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9610000000000001e-05 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:30:57.974 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:30:57.975 INFO:teuthology.orchestra.run.smithi007.stdout:3005 clone 31 order 23 su 524288 sc 3 2024-09-12T09:30:57.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:30:57.977 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:57.975+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:57.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:30:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:30:58.008 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:30:58.008 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:30:58.025 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:30:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:30:58.042 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:30:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:30:58.063 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:30:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:30:58.077 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:58.075+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:58.087 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:30:58.088 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:30:58.111 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.111 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.112 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:30:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:30:58.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:30:58.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:30:58.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:30:58.178 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:58.175+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:58.274 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.274 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.279 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:58.275+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:58.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:30:58.296 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-12T09:30:58.321 INFO:teuthology.orchestra.run.smithi007.stdout:3006 write 0x1f48fc2 thru 0x1f57b1d (0xeb5c bytes) 2024-09-12T09:30:58.337 INFO:teuthology.orchestra.run.smithi007.stdout:3008 read 0xb8c65a thru 0xb8e4ae (0x1e55 bytes) 2024-09-12T09:30:58.349 INFO:teuthology.orchestra.run.smithi007.stdout:3009 trunc from 0x24e3f76 to 0x9fa4977 2024-09-12T09:30:58.361 INFO:teuthology.orchestra.run.smithi007.stdout:3011 read 0x9c80dcd thru 0x9c892ef (0x8523 bytes) 2024-09-12T09:30:58.361 INFO:teuthology.orchestra.run.smithi007.stdout:3012 punch from 0x212c263 to 0x21315e6, (0x5383 bytes) 2024-09-12T09:30:58.361 INFO:teuthology.orchestra.run.smithi007.stdout:3013 write 0x44933be thru 0x44a2e28 (0xfa6b bytes) 2024-09-12T09:30:58.369 INFO:teuthology.orchestra.run.smithi007.stdout:3015 write 0x6eae661 thru 0x6eba634 (0xbfd4 bytes) 2024-09-12T09:30:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:3016 write 0x6285d40 thru 0x62950a5 (0xf366 bytes) 2024-09-12T09:30:58.378 INFO:teuthology.orchestra.run.smithi007.stdout:3017 read 0x1cf0f1f thru 0x1cf4cb8 (0x3d9a bytes) 2024-09-12T09:30:58.379 INFO:teuthology.orchestra.run.smithi007.stdout:3018 write 0x8b9b883 thru 0x8b9dc6a (0x23e8 bytes) 2024-09-12T09:30:58.380 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:30:58.376+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:30:58.385 INFO:teuthology.orchestra.run.smithi007.stdout:3019 read 0x12cec51 thru 0x12d8578 (0x9928 bytes) 2024-09-12T09:30:58.386 INFO:teuthology.orchestra.run.smithi007.stdout:3020 read 0x99573e3 thru 0x995a33f (0x2f5d bytes) 2024-09-12T09:30:58.386 INFO:teuthology.orchestra.run.smithi007.stdout:3021 punch from 0x7c56162 to 0x7c5ea99, (0x8937 bytes) 2024-09-12T09:30:58.386 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:58.401 INFO:teuthology.orchestra.run.smithi007.stdout:3024 write 0xbb39b6f thru 0xbb47276 (0xd708 bytes) 2024-09-12T09:30:58.408 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:58.408 INFO:teuthology.orchestra.run.smithi007.stdout:3026 read 0x52fbcb8 thru 0x5300a30 (0x4d79 bytes) 2024-09-12T09:30:58.419 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.419 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.423 INFO:teuthology.orchestra.run.smithi007.stdout:3031 write 0xdbd3f5f thru 0xdbd4cbc (0xd5e bytes) 2024-09-12T09:30:58.431 INFO:teuthology.orchestra.run.smithi007.stdout:3032 writesame 0xa885f8 thru 0xa88d14 data_size 0x1(0x71d bytes) 2024-09-12T09:30:58.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:30:58.465 INFO:teuthology.orchestra.run.smithi007.stdout:3033 trunc from 0xdbd4cbd to 0xe62e817 2024-09-12T09:30:58.477 INFO:teuthology.orchestra.run.smithi007.stdout:3034 write 0xc367a84 thru 0xc368544 (0xac1 bytes) 2024-09-12T09:30:58.482 INFO:teuthology.orchestra.run.smithi007.stdout:3035 write 0x2ed550 thru 0x2f5448 (0x7ef9 bytes) 2024-09-12T09:30:58.484 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:58.483+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:58.506 INFO:teuthology.orchestra.run.smithi007.stdout:3036 write 0x5f5a5b4 thru 0x5f66ba1 (0xc5ee bytes) 2024-09-12T09:30:58.548 INFO:teuthology.orchestra.run.smithi007.stdout:3037 trunc from 0xe62e817 to 0xb977c8 2024-09-12T09:30:58.581 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:30:58.579+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:30:58.584 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.584 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.597 INFO:teuthology.orchestra.run.smithi007.stdout:3038 trunc from 0xb977c8 to 0x24a6260 2024-09-12T09:30:58.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:30:58.650 INFO:teuthology.orchestra.run.smithi007.stdout:3039 write 0xdfc7e43 thru 0xdfd4f47 (0xd105 bytes) 2024-09-12T09:30:58.656 INFO:teuthology.orchestra.run.smithi007.stdout:3040 read 0x481bafc thru 0x481c605 (0xb0a bytes) 2024-09-12T09:30:58.656 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:58.656 INFO:teuthology.orchestra.run.smithi007.stdout:3043 clone 32 order 22 su 65536 sc 8 2024-09-12T09:30:58.684 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:58.683+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:58.763 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.763 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:30:58.782 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:58.780+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:58.883 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:58.879+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:58.945 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:58.945 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:58.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:30:58.984 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:30:58.984+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:30:59.085 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:30:59.083+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:30:59.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:30:59.130 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.130 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:30:59.185 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:59.184+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:59.212 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone31 from 0xdfd4f48 (overlap 0xb977c8) to 0x128bfa 2024-09-12T09:30:59.286 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:59.284+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:59.288 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-12T09:30:59.314 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.314 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:30:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:3044 writesame 0x897dba9 thru 0x898bb56 data_size 0x1(0xdfae bytes) 2024-09-12T09:30:59.387 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:59.384+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:59.400 INFO:teuthology.orchestra.run.smithi007.stdout:3045 write 0x1d7194 thru 0x1e1306 (0xa173 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3046 read 0x6f49441 thru 0x6f4f6e5 (0x62a5 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3047 read 0xb82c9dd thru 0xb82dcbf (0x12e3 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3048 read 0xb793fd6 thru 0xb7940e9 (0x114 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3051 read 0x1812afe thru 0x181cc51 (0xa154 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3052 read 0xc9c55aa thru 0xc9d0499 (0xaef0 bytes) 2024-09-12T09:30:59.428 INFO:teuthology.orchestra.run.smithi007.stdout:3056 write 0x5c91ae1 thru 0x5c98aa0 (0x6fc0 bytes) 2024-09-12T09:30:59.434 INFO:teuthology.orchestra.run.smithi007.stdout:3058 read 0x7d992e1 thru 0x7da2784 (0x94a4 bytes) 2024-09-12T09:30:59.435 INFO:teuthology.orchestra.run.smithi007.stdout:3060 punch from 0xa265b54 to 0xa267bf8, (0x20a4 bytes) 2024-09-12T09:30:59.435 INFO:teuthology.orchestra.run.smithi007.stdout:3061 punch from 0xc545d78 to 0xc554e98, (0xf120 bytes) 2024-09-12T09:30:59.435 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:59.435 INFO:teuthology.orchestra.run.smithi007.stdout:3065 read 0x5e46414 thru 0x5e4ef3c (0x8b29 bytes) 2024-09-12T09:30:59.440 INFO:teuthology.orchestra.run.smithi007.stdout:3066 trunc from 0xdfd4f48 to 0xac4e927 2024-09-12T09:30:59.464 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.464 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:30:59.484 INFO:teuthology.orchestra.run.smithi007.stdout:3067 write 0xe89eb7b thru 0xe89f8f2 (0xd78 bytes) 2024-09-12T09:30:59.492 INFO:teuthology.orchestra.run.smithi007.stdout:3068 read 0xdefe981 thru 0xdeff075 (0x6f5 bytes) 2024-09-12T09:30:59.492 INFO:teuthology.orchestra.run.smithi007.stdout:3069 read 0x2051d4c thru 0x2059ef4 (0x81a9 bytes) 2024-09-12T09:30:59.493 INFO:teuthology.orchestra.run.smithi007.stdout:3070 write 0x89d84c8 thru 0x89dbdb1 (0x38ea bytes) 2024-09-12T09:30:59.497 INFO:teuthology.orchestra.run.smithi007.stdout:3071 punch from 0x7cbf6b4 to 0x7cc5901, (0x624d bytes) 2024-09-12T09:30:59.498 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:59.498 INFO:teuthology.orchestra.run.smithi007.stdout:3073 writesame 0xd2f8156 thru 0xd30454b data_size 0x1b(0xc3f6 bytes) 2024-09-12T09:30:59.505 INFO:teuthology.orchestra.run.smithi007.stdout:3076 writesame 0x9f688c1 thru 0x9f70649 data_size 0x1(0x7d89 bytes) 2024-09-12T09:30:59.528 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:59.527+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:59.528 INFO:teuthology.orchestra.run.smithi007.stdout:3077 punch from 0x4f55f23 to 0x4f59e71, (0x3f4e bytes) 2024-09-12T09:30:59.549 INFO:teuthology.orchestra.run.smithi007.stdout:3078 trunc from 0xe89f8f3 to 0x55f1f8b 2024-09-12T09:30:59.588 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:30:59.587+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:30:59.601 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.601 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:30:59.621 INFO:teuthology.orchestra.run.smithi007.stdout:3079 writesame 0xa0789d7 thru 0xa07cbae data_size 0x2b(0x41d8 bytes) 2024-09-12T09:30:59.621 INFO:teuthology.orchestra.run.smithi007.stdout:3080 write 0xff72d0 thru 0xffd1e0 (0x5f11 bytes) 2024-09-12T09:30:59.639 INFO:teuthology.orchestra.run.smithi007.stdout:3084 read 0x11d7449 thru 0x11dd2f8 (0x5eb0 bytes) 2024-09-12T09:30:59.640 INFO:teuthology.orchestra.run.smithi007.stdout:3085 read 0x4c23782 thru 0x4c2ebdf (0xb45e bytes) 2024-09-12T09:30:59.641 INFO:teuthology.orchestra.run.smithi007.stdout:3086 write 0x95021ce thru 0x9508535 (0x6368 bytes) 2024-09-12T09:30:59.649 INFO:teuthology.orchestra.run.smithi007.stdout:3087 writesame 0x2d29e38 thru 0x2d31ec2 data_size 0x7(0x808b bytes) 2024-09-12T09:30:59.659 INFO:teuthology.orchestra.run.smithi007.stdout:3088 writesame 0x6f0b2bd thru 0x6f1923c data_size 0x40(0xdf80 bytes) 2024-09-12T09:30:59.693 INFO:teuthology.orchestra.run.smithi007.stdout:3089 write 0xebb9624 thru 0xebc5049 (0xba26 bytes) 2024-09-12T09:30:59.702 INFO:teuthology.orchestra.run.smithi007.stdout:3090 write 0x4a80941 thru 0x4a8fc2a (0xf2ea bytes) 2024-09-12T09:30:59.704 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:30:59.703+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:30:59.705 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:30:59.705 INFO:teuthology.orchestra.run.smithi007.stdout:3093 read 0x6f7c451 thru 0x6f7d65c (0x120c bytes) 2024-09-12T09:30:59.705 INFO:teuthology.orchestra.run.smithi007.stdout:3095 clone 33 order 18 su 1024 sc 10 2024-09-12T09:30:59.782 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.782 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.790 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:30:59.788+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:30:59.796 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:30:59.891 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:30:59.888+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:30:59.962 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:30:59.962 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:30:59.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:00.002 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 0, 7] out_osds: [6, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:00.002 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:00.002 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T09:31:00.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:31:00.020 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:00.019+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:00.093 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:00.092+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:00.094 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:00.094 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:00.109 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:00.110 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:31:00.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:00.196 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:00.195+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:00.202 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone32 from 0xebc504a (overlap 0x55f1f8b) to 0x4d59c63 2024-09-12T09:31:00.279 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:00.279 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:00.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:00.295 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:00.292+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:00.334 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:00.335 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:00.342 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-12T09:31:00.348 INFO:teuthology.orchestra.run.smithi007.stdout:3096 read 0x612f2ad thru 0x613e83a (0xf58e bytes) 2024-09-12T09:31:00.349 INFO:teuthology.orchestra.run.smithi007.stdout:3098 read 0x1b46232 thru 0x1b48418 (0x21e7 bytes) 2024-09-12T09:31:00.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:00.353 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 96, 2024-09-12T09:31:00.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:31:00.353 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:31:00.102956+0000", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:29:16.082807+0000", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:30:42.933178+0000", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:31:00.354 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:31:00.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:00.356 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:00.357 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:31:00.358 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:00.359 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:00.360 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.361 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.362 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:00.363 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:00.364 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:00.365 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:00.367 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:31:00.368 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:00.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:31:00.370 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "96", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:00.371 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 35, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 96, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:00.372 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:00.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.75, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.75, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.309999942779541, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.309999942779541, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 83, 2024-09-12T09:31:00.376 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:31:00.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.381 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.382 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 66, 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:31:00.387 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.388 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 83, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:31:00.389 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:31:00.393 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.394 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 83, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:31:00.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:31:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.400 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 83, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:31:00.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.407 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 80, 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:31:00.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:31:00.412 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:31:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:31:00.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:31:00.418 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:00.419 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 80, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.421 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:00.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.425 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:00.426 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.427 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:31:00.430 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:31:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.2", 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:31:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:31:00.435 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:31:00.436 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:31:00.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.436 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:00.436 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:00.437 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:00.437 INFO:teuthology.orchestra.run.smithi007.stdout:3099 trunc from 0xebc504a to 0x6dd8bf 2024-09-12T09:31:00.437 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:00.438 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:00.439 INFO:tasks.thrashosds.thrasher:j is {'epoch': 96, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:31:00.102956+0000', 'last_up_change': '2024-09-12T09:29:16.082807+0000', 'last_in_change': '2024-09-12T09:30:42.933178+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [6, 7]}, {'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:31:00.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:31:00.440 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:00.440 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:00.435+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:00.444 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:00.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:00.503 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:00.503 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:00.511 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:00.511 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:00.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:00.529 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:00.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:00.572 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:00.571+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:00.638 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:00.635+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149783 3.1f 3.eff70c5f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.671014+0000", 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0067000000000001e-05, 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3808999999999998e-05, 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:00.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149783 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671014+0000", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671014+0000", 2024-09-12T09:31:00.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671015+0000", 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5999999999999991e-07 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671019+0000", 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.386e-06 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671028+0000", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4839999999999993e-06 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671033+0000", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7330000000000003e-06 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.671076+0000", 2024-09-12T09:31:00.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2939999999999999e-05 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:00.675 INFO:teuthology.orchestra.run.smithi007.stdout:3102 write 0xea45e11 thru 0xea553d9 (0xf5c9 bytes) 2024-09-12T09:31:00.691 INFO:teuthology.orchestra.run.smithi007.stdout:3103 punch from 0xb5576f0 to 0xb55885a, (0x116a bytes) 2024-09-12T09:31:00.691 INFO:teuthology.orchestra.run.smithi007.stdout:3105 punch from 0xac150d7 to 0xac1944a, (0x4373 bytes) 2024-09-12T09:31:00.692 INFO:teuthology.orchestra.run.smithi007.stdout:3106 writesame 0xdc8af2f thru 0xdc99d1e data_size 0x1b(0xedf0 bytes) 2024-09-12T09:31:00.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:00.710 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.710 INFO:teuthology.orchestra.run.smithi007.stdout:3108 write 0x1a05cb1 thru 0x1a0bae4 (0x5e34 bytes) 2024-09-12T09:31:00.728 INFO:teuthology.orchestra.run.smithi007.stdout:3109 read 0x76a5f32 thru 0x76a7e54 (0x1f23 bytes) 2024-09-12T09:31:00.728 INFO:teuthology.orchestra.run.smithi007.stdout:3110 writesame 0xd79ca93 thru 0xd7ab60c data_size 0x99(0xeb7a bytes) 2024-09-12T09:31:00.739 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:00.735+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:00.748 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.748 INFO:teuthology.orchestra.run.smithi007.stdout:3112 read 0x42d242 thru 0x435978 (0x8737 bytes) 2024-09-12T09:31:00.767 INFO:teuthology.orchestra.run.smithi007.stdout:3113 write 0xb5817a3 thru 0xb58d248 (0xbaa6 bytes) 2024-09-12T09:31:00.784 INFO:teuthology.orchestra.run.smithi007.stdout:3114 write 0x4832f86 thru 0x483f0b9 (0xc134 bytes) 2024-09-12T09:31:00.799 INFO:teuthology.orchestra.run.smithi007.stdout:3115 write 0x3dc2aa3 thru 0x3dc5121 (0x267f bytes) 2024-09-12T09:31:00.816 INFO:teuthology.orchestra.run.smithi007.stdout:3116 punch from 0x4999ffb to 0x499b447, (0x144c bytes) 2024-09-12T09:31:00.817 INFO:teuthology.orchestra.run.smithi007.stdout:3117 read 0xb5e15fd thru 0xb5e3ec9 (0x28cd bytes) 2024-09-12T09:31:00.817 INFO:teuthology.orchestra.run.smithi007.stdout:3118 punch from 0xcfb8376 to 0xcfc5bc3, (0xd84d bytes) 2024-09-12T09:31:00.817 INFO:teuthology.orchestra.run.smithi007.stdout:3120 read 0x365779b thru 0x3658e13 (0x1679 bytes) 2024-09-12T09:31:00.817 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.818 INFO:teuthology.orchestra.run.smithi007.stdout:3125 punch from 0xd5e4c81 to 0xd5ed8ca, (0x8c49 bytes) 2024-09-12T09:31:00.818 INFO:teuthology.orchestra.run.smithi007.stdout:3126 write 0x1f09e89 thru 0x1f14fa4 (0xb11c bytes) 2024-09-12T09:31:00.830 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:00.831 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:00.834 INFO:teuthology.orchestra.run.smithi007.stdout:3127 writesame 0x566fc2b thru 0x567cca0 data_size 0x1(0xd076 bytes) 2024-09-12T09:31:00.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:31:00.840 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:00.839+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:00.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:00.857 INFO:teuthology.orchestra.run.smithi007.stdout:3128 write 0x5fa1d96 thru 0x5fa2cfe (0xf69 bytes) 2024-09-12T09:31:00.861 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-12T09:31:00.111839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1306065438,"num_objects":1308,"num_object_clones":305,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":490,"num_read":171388,"num_read_kb":1206045,"num_write":10125,"num_write_kb":703287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":347624051,"num_keys_recovered":207,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7816,"ondisk_log_size":7816,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":285944,"kb_used_data":73776,"kb_used_omap":9,"kb_used_meta":211958,"kb_avail":565945096,"statfs":{"total":579820584960,"available":579527778304,"internally_reserved":0,"allocated":75546624,"data_stored":985583444,"data_compressed":9868136,"data_compressed_allocated":61456384,"data_compressed_original":977506152,"omap_allocated":9563,"internal_metadata":217045669},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20755043,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":1700,"num_read_kb":3355,"num_write":295,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033155"},"pg_stats":[{"pgid":"3.1f","version":"91'429","reported_seq":4892,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.656547+0000","last_change":"2024-09-12T09:29:58.167241+0000","last_active":"2024-09-12T09:30:55.656547+0000","last_peered":"2024-09-12T09:30:55.656547+0000","last_clean":"2024-09-12T09:30:55.656547+0000","last_became_active":"2024-09-12T09:29:58.165903+0000","last_became_peered":"2024-09-12T09:29:58.165903+0000","last_unstale":"2024-09-12T09:30:55.656547+0000","last_undegraded":"2024-09-12T09:30:55.656547+0000","last_fullsized":"2024-09-12T09:30:55.656547+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:15.357170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":86696197,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":15,"num_read":5574,"num_read_kb":88659,"num_write":597,"num_write_kb":45447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":4329673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"91'416","reported_seq":4197,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.851366+0000","last_change":"2024-09-12T09:30:01.194793+0000","last_active":"2024-09-12T09:30:55.851366+0000","last_peered":"2024-09-12T09:30:55.851366+0000","last_clean":"2024-09-12T09:30:55.851366+0000","last_became_active":"2024-09-12T09:30:01.169430+0000","last_became_peered":"2024-09-12T09:30:01.169430+0000","last_unstale":"2024-09-12T09:30:55.851366+0000","last_undegraded":"2024-09-12T09:30:55.851366+0000","last_fullsized":"2024-09-12T09:30:55.851366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:07.034189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":41893710,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":5974,"num_read_kb":34185,"num_write":571,"num_write_kb":33806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":352930,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"91'205","reported_seq":3889,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.753790+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:30:55.753790+0000","last_peered":"2024-09-12T09:30:55.753790+0000","last_clean":"2024-09-12T09:30:55.753790+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:30:55.753790+0000","last_undegraded":"2024-09-12T09:30:55.753790+0000","last_fullsized":"2024-09-12T09:30:55.753790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:50:29.053461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":43720807,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":4375,"num_read_kb":66120,"num_write":186,"num_write_kb":36021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'196","reported_seq":4042,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.792612+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:30:55.792612+0000","last_peered":"2024-09-12T09:30:55.792612+0000","last_clean":"2024-09-12T09:30:55.792612+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:30:55.792612+0000","last_undegraded":"2024-09-12T09:30:55.792612+0000","last_fullsized":"2024-09-12T09:30:55.792612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:35:51.139995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":20005207,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":17,"num_read":4549,"num_read_kb":34080,"num_write":211,"num_write_kb":11635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'179","reported_seq":4845,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.884598+0000","last_change":"2024-09-12T09:29:55.231536+0000","last_active":"2024-09-12T09:30:55.884598+0000","last_peered":"2024-09-12T09:30:55.884598+0000","last_clean":"2024-09-12T09:30:55.884598+0000","last_became_active":"2024-09-12T09:29:55.137238+0000","last_became_peered":"2024-09-12T09:29:55.137238+0000","last_unstale":"2024-09-12T09:30:55.884598+0000","last_undegraded":"2024-09-12T09:30:55.884598+0000","last_fullsized":"2024-09-12T09:30:55.884598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:07.904133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":27535922,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":5307,"num_read_kb":13428,"num_write":188,"num_write_kb":4268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1206936,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"92'244","reported_seq":2495,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:57.051907+0000","last_change":"2024-09-12T09:30:46.979355+0000","last_active":"2024-09-12T09:30:57.051907+0000","last_peered":"2024-09-12T09:30:57.051907+0000","last_clean":"2024-09-12T09:30:57.051907+0000","last_became_active":"2024-09-12T09:30:46.978472+0000","last_became_peered":"2024-09-12T09:30:46.978472+0000","last_unstale":"2024-09-12T09:30:57.051907+0000","last_undegraded":"2024-09-12T09:30:57.051907+0000","last_fullsized":"2024-09-12T09:30:57.051907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:50.609059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":9296325,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":2636,"num_read_kb":9756,"num_write":267,"num_write_kb":30157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2195003,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"91'207","reported_seq":830,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.630583+0000","last_change":"2024-09-12T09:30:44.080862+0000","last_active":"2024-09-12T09:30:55.630583+0000","last_peered":"2024-09-12T09:30:55.630583+0000","last_clean":"2024-09-12T09:30:55.630583+0000","last_became_active":"2024-09-12T09:30:43.967238+0000","last_became_peered":"2024-09-12T09:30:43.967238+0000","last_unstale":"2024-09-12T09:30:55.630583+0000","last_undegraded":"2024-09-12T09:30:55.630583+0000","last_fullsized":"2024-09-12T09:30:55.630583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:17:55.027272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":24890588,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":1118,"num_read_kb":4578,"num_write":256,"num_write_kb":2012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":970649,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"91'157","reported_seq":1317,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.997066+0000","last_change":"2024-09-12T09:29:55.177304+0000","last_active":"2024-09-12T09:30:55.997066+0000","last_peered":"2024-09-12T09:30:55.997066+0000","last_clean":"2024-09-12T09:30:55.997066+0000","last_became_active":"2024-09-12T09:29:55.134985+0000","last_became_peered":"2024-09-12T09:29:55.134985+0000","last_unstale":"2024-09-12T09:30:55.997066+0000","last_undegraded":"2024-09-12T09:30:55.997066+0000","last_fullsized":"2024-09-12T09:30:55.997066+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:56:50.116445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":11982608,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":1176,"num_read_kb":5400,"num_write":167,"num_write_kb":4897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":524806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'173","reported_seq":2703,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.065228+0000","last_change":"2024-09-12T09:29:55.167682+0000","last_active":"2024-09-12T09:30:55.065228+0000","last_peered":"2024-09-12T09:30:55.065228+0000","last_clean":"2024-09-12T09:30:55.065228+0000","last_became_active":"2024-09-12T09:29:55.136789+0000","last_became_peered":"2024-09-12T09:29:55.136789+0000","last_unstale":"2024-09-12T09:30:55.065228+0000","last_undegraded":"2024-09-12T09:30:55.065228+0000","last_fullsized":"2024-09-12T09:30:55.065228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:17.955073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":42661648,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":3206,"num_read_kb":47503,"num_write":269,"num_write_kb":36402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8413,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"91'241","reported_seq":13732,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.979033+0000","last_change":"2024-09-12T09:30:33.669800+0000","last_active":"2024-09-12T09:30:55.979033+0000","last_peered":"2024-09-12T09:30:55.979033+0000","last_clean":"2024-09-12T09:30:55.979033+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:30:55.979033+0000","last_undegraded":"2024-09-12T09:30:55.979033+0000","last_fullsized":"2024-09-12T09:30:55.979033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:53.300421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":63785980,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":13541,"num_read_kb":70071,"num_write":319,"num_write_kb":27292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"92'495","reported_seq":5633,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.253977+0000","last_change":"2024-09-12T09:30:34.672921+0000","last_active":"2024-09-12T09:30:56.253977+0000","last_peered":"2024-09-12T09:30:56.253977+0000","last_clean":"2024-09-12T09:30:56.253977+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:30:56.253977+0000","last_undegraded":"2024-09-12T09:30:56.253977+0000","last_fullsized":"2024-09-12T09:30:56.253977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":495,"log_dups_size":0,"ondisk_log_size":495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:20:57.093682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":83549485,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":23,"num_read":6568,"num_read_kb":77517,"num_write":747,"num_write_kb":42377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"92'529","reported_seq":8268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.081874+0000","last_change":"2024-09-12T09:30:33.668610+0000","last_active":"2024-09-12T09:30:56.081874+0000","last_peered":"2024-09-12T09:30:56.081874+0000","last_clean":"2024-09-12T09:30:56.081874+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:30:56.081874+0000","last_undegraded":"2024-09-12T09:30:56.081874+0000","last_fullsized":"2024-09-12T09:30:56.081874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:12.781321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":30278949,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":9703,"num_read_kb":20401,"num_write":868,"num_write_kb":23555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.085511+0000","last_change":"2024-09-12T09:30:30.871108+0000","last_active":"2024-09-12T09:30:56.085511+0000","last_peered":"2024-09-12T09:30:56.085511+0000","last_clean":"2024-09-12T09:30:56.085511+0000","last_became_active":"2024-09-12T09:30:30.870346+0000","last_became_peered":"2024-09-12T09:30:30.870346+0000","last_unstale":"2024-09-12T09:30:56.085511+0000","last_undegraded":"2024-09-12T09:30:56.085511+0000","last_fullsized":"2024-09-12T09:30:56.085511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:23.248076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"91'136","reported_seq":4972,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.978289+0000","last_change":"2024-09-12T09:30:46.982993+0000","last_active":"2024-09-12T09:30:55.978289+0000","last_peered":"2024-09-12T09:30:55.978289+0000","last_clean":"2024-09-12T09:30:55.978289+0000","last_became_active":"2024-09-12T09:30:46.981203+0000","last_became_peered":"2024-09-12T09:30:46.981203+0000","last_unstale":"2024-09-12T09:30:55.978289+0000","last_undegraded":"2024-09-12T09:30:55.978289+0000","last_fullsized":"2024-09-12T09:30:55.978289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:47.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":21420921,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":5484,"num_read_kb":20196,"num_write":119,"num_write_kb":5197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":4829848,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.082009+0000","last_change":"2024-09-12T09:30:43.963886+0000","last_active":"2024-09-12T09:30:56.082009+0000","last_peered":"2024-09-12T09:30:56.082009+0000","last_clean":"2024-09-12T09:30:56.082009+0000","last_became_active":"2024-09-12T09:30:43.962849+0000","last_became_peered":"2024-09-12T09:30:43.962849+0000","last_unstale":"2024-09-12T09:30:56.082009+0000","last_undegraded":"2024-09-12T09:30:56.082009+0000","last_fullsized":"2024-09-12T09:30:56.082009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:38.942689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"92'160","reported_seq":2095,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.890185+0000","last_change":"2024-09-12T09:30:33.673235+0000","last_active":"2024-09-12T09:30:56.890185+0000","last_peered":"2024-09-12T09:30:56.890185+0000","last_clean":"2024-09-12T09:30:56.890185+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:30:56.890185+0000","last_undegraded":"2024-09-12T09:30:56.890185+0000","last_fullsized":"2024-09-12T09:30:56.890185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:02.301387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":25341962,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":1818,"num_read_kb":14654,"num_write":219,"num_write_kb":22543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.085680+0000","last_change":"2024-09-12T09:30:30.875450+0000","last_active":"2024-09-12T09:30:56.085680+0000","last_peered":"2024-09-12T09:30:56.085680+0000","last_clean":"2024-09-12T09:30:56.085680+0000","last_became_active":"2024-09-12T09:30:30.874655+0000","last_became_peered":"2024-09-12T09:30:30.874655+0000","last_unstale":"2024-09-12T09:30:56.085680+0000","last_undegraded":"2024-09-12T09:30:56.085680+0000","last_fullsized":"2024-09-12T09:30:56.085680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:30.720674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"91'378","reported_seq":5184,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.994446+0000","last_change":"2024-09-12T09:30:46.985105+0000","last_active":"2024-09-12T09:30:55.994446+0000","last_peered":"2024-09-12T09:30:55.994446+0000","last_clean":"2024-09-12T09:30:55.994446+0000","last_became_active":"2024-09-12T09:30:46.983676+0000","last_became_peered":"2024-09-12T09:30:46.983676+0000","last_unstale":"2024-09-12T09:30:55.994446+0000","last_undegraded":"2024-09-12T09:30:55.994446+0000","last_fullsized":"2024-09-12T09:30:55.994446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:57.477189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":25773600,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":5819,"num_read_kb":24245,"num_write":650,"num_write_kb":45299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":7432750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:54.068000+0000","last_change":"2024-09-12T09:30:43.961453+0000","last_active":"2024-09-12T09:30:54.068000+0000","last_peered":"2024-09-12T09:30:54.068000+0000","last_clean":"2024-09-12T09:30:54.068000+0000","last_became_active":"2024-09-12T09:30:43.960994+0000","last_became_peered":"2024-09-12T09:30:43.960994+0000","last_unstale":"2024-09-12T09:30:54.068000+0000","last_undegraded":"2024-09-12T09:30:54.068000+0000","last_fullsized":"2024-09-12T09:30:54.068000+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.668460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"91'150","reported_seq":13378,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.977994+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:30:55.977994+0000","last_peered":"2024-09-12T09:30:55.977994+0000","last_clean":"2024-09-12T09:30:55.977994+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:30:55.977994+0000","last_undegraded":"2024-09-12T09:30:55.977994+0000","last_fullsized":"2024-09-12T09:30:55.977994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:53.908557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":36016372,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":13355,"num_read_kb":55550,"num_write":132,"num_write_kb":11462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"92'338","reported_seq":4279,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.223273+0000","last_change":"2024-09-12T09:29:55.204077+0000","last_active":"2024-09-12T09:30:56.223273+0000","last_peered":"2024-09-12T09:30:56.223273+0000","last_clean":"2024-09-12T09:30:56.223273+0000","last_became_active":"2024-09-12T09:29:55.135753+0000","last_became_peered":"2024-09-12T09:29:55.135753+0000","last_unstale":"2024-09-12T09:30:56.223273+0000","last_undegraded":"2024-09-12T09:30:56.223273+0000","last_fullsized":"2024-09-12T09:30:56.223273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:24:07.186408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":25344016,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":24,"num_read":5190,"num_read_kb":19677,"num_write":410,"num_write_kb":9455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":57888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:54.073532+0000","last_change":"2024-09-12T09:30:43.969129+0000","last_active":"2024-09-12T09:30:54.073532+0000","last_peered":"2024-09-12T09:30:54.073532+0000","last_clean":"2024-09-12T09:30:54.073532+0000","last_became_active":"2024-09-12T09:30:43.962175+0000","last_became_peered":"2024-09-12T09:30:43.962175+0000","last_unstale":"2024-09-12T09:30:54.073532+0000","last_undegraded":"2024-09-12T09:30:54.073532+0000","last_fullsized":"2024-09-12T09:30:54.073532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:00:15.922849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'181","reported_seq":3524,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.747723+0000","last_change":"2024-09-12T09:30:46.984392+0000","last_active":"2024-09-12T09:30:55.747723+0000","last_peered":"2024-09-12T09:30:55.747723+0000","last_clean":"2024-09-12T09:30:55.747723+0000","last_became_active":"2024-09-12T09:30:46.982399+0000","last_became_peered":"2024-09-12T09:30:46.982399+0000","last_unstale":"2024-09-12T09:30:55.747723+0000","last_undegraded":"2024-09-12T09:30:55.747723+0000","last_fullsized":"2024-09-12T09:30:55.747723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:05.498172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":45549487,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":3127,"num_read_kb":34218,"num_write":154,"num_write_kb":14595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":2593871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:54.067149+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:30:54.067149+0000","last_peered":"2024-09-12T09:30:54.067149+0000","last_clean":"2024-09-12T09:30:54.067149+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:30:54.067149+0000","last_undegraded":"2024-09-12T09:30:54.067149+0000","last_fullsized":"2024-09-12T09:30:54.067149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'191","reported_seq":9751,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.883732+0000","last_change":"2024-09-12T09:29:55.163561+0000","last_active":"2024-09-12T09:30:55.883732+0000","last_peered":"2024-09-12T09:30:55.883732+0000","last_clean":"2024-09-12T09:30:55.883732+0000","last_became_active":"2024-09-12T09:29:55.134480+0000","last_became_peered":"2024-09-12T09:29:55.134480+0000","last_unstale":"2024-09-12T09:30:55.883732+0000","last_undegraded":"2024-09-12T09:30:55.883732+0000","last_fullsized":"2024-09-12T09:30:55.883732+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:39.456957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":39025076,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":10445,"num_read_kb":33035,"num_write":168,"num_write_kb":20175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2206098,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.085215+0000","last_change":"2024-09-12T09:30:43.963299+0000","last_active":"2024-09-12T09:30:56.085215+0000","last_peered":"2024-09-12T09:30:56.085215+0000","last_clean":"2024-09-12T09:30:56.085215+0000","last_became_active":"2024-09-12T09:30:43.961820+0000","last_became_peered":"2024-09-12T09:30:43.961820+0000","last_unstale":"2024-09-12T09:30:56.085215+0000","last_undegraded":"2024-09-12T09:30:56.085215+0000","last_fullsized":"2024-09-12T09:30:56.085215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:20.604956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'147","reported_seq":4735,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.935913+0000","last_change":"2024-09-12T09:30:12.711941+0000","last_active":"2024-09-12T09:30:55.935913+0000","last_peered":"2024-09-12T09:30:55.935913+0000","last_clean":"2024-09-12T09:30:55.935913+0000","last_became_active":"2024-09-12T09:30:12.711307+0000","last_became_peered":"2024-09-12T09:30:12.711307+0000","last_unstale":"2024-09-12T09:30:55.935913+0000","last_undegraded":"2024-09-12T09:30:55.935913+0000","last_fullsized":"2024-09-12T09:30:55.935913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:43.573884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":33256095,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":5619,"num_read_kb":42413,"num_write":163,"num_write_kb":10838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":160,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:54.073638+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:30:54.073638+0000","last_peered":"2024-09-12T09:30:54.073638+0000","last_clean":"2024-09-12T09:30:54.073638+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:30:54.073638+0000","last_undegraded":"2024-09-12T09:30:54.073638+0000","last_fullsized":"2024-09-12T09:30:54.073638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.085012+0000","last_change":"2024-09-12T09:30:30.889337+0000","last_active":"2024-09-12T09:30:56.085012+0000","last_peered":"2024-09-12T09:30:56.085012+0000","last_clean":"2024-09-12T09:30:56.085012+0000","last_became_active":"2024-09-12T09:30:30.873745+0000","last_became_peered":"2024-09-12T09:30:30.873745+0000","last_unstale":"2024-09-12T09:30:56.085012+0000","last_undegraded":"2024-09-12T09:30:56.085012+0000","last_fullsized":"2024-09-12T09:30:56.085012+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:57.848261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'243","reported_seq":3932,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.978820+0000","last_change":"2024-09-12T09:30:46.981798+0000","last_active":"2024-09-12T09:30:55.978820+0000","last_peered":"2024-09-12T09:30:55.978820+0000","last_clean":"2024-09-12T09:30:55.978820+0000","last_became_active":"2024-09-12T09:30:46.980108+0000","last_became_peered":"2024-09-12T09:30:46.980108+0000","last_unstale":"2024-09-12T09:30:55.978820+0000","last_undegraded":"2024-09-12T09:30:55.978820+0000","last_fullsized":"2024-09-12T09:30:55.978820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:21:25.737331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":39055600,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":5350,"num_read_kb":49165,"num_write":289,"num_write_kb":20867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":10746792,"num_keys_recovered":21,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"92'196","reported_seq":2118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.903956+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:30:56.903956+0000","last_peered":"2024-09-12T09:30:56.903956+0000","last_clean":"2024-09-12T09:30:56.903956+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:30:56.903956+0000","last_undegraded":"2024-09-12T09:30:56.903956+0000","last_fullsized":"2024-09-12T09:30:56.903956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:40:21.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":32550793,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":2000,"num_read_kb":11186,"num_write":215,"num_write_kb":9057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'204","reported_seq":4852,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.890393+0000","last_change":"2024-09-12T09:30:46.979031+0000","last_active":"2024-09-12T09:30:56.890393+0000","last_peered":"2024-09-12T09:30:56.890393+0000","last_clean":"2024-09-12T09:30:56.890393+0000","last_became_active":"2024-09-12T09:30:46.977847+0000","last_became_peered":"2024-09-12T09:30:46.977847+0000","last_unstale":"2024-09-12T09:30:56.890393+0000","last_undegraded":"2024-09-12T09:30:56.890393+0000","last_fullsized":"2024-09-12T09:30:56.890393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:37.980266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":29778382,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":4725,"num_read_kb":11488,"num_write":277,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":7309669,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'294","reported_seq":5568,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.887564+0000","last_change":"2024-09-12T09:29:55.239142+0000","last_active":"2024-09-12T09:30:55.887564+0000","last_peered":"2024-09-12T09:30:55.887564+0000","last_clean":"2024-09-12T09:30:55.887564+0000","last_became_active":"2024-09-12T09:29:55.135093+0000","last_became_peered":"2024-09-12T09:29:55.135093+0000","last_unstale":"2024-09-12T09:30:55.887564+0000","last_undegraded":"2024-09-12T09:30:55.887564+0000","last_fullsized":"2024-09-12T09:30:55.887564+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:12.773550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":73662533,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":5881,"num_read_kb":70738,"num_write":476,"num_write_kb":31723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1338871,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'164","reported_seq":2271,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.920096+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:30:55.920096+0000","last_peered":"2024-09-12T09:30:55.920096+0000","last_clean":"2024-09-12T09:30:55.920096+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:30:55.920096+0000","last_undegraded":"2024-09-12T09:30:55.920096+0000","last_fullsized":"2024-09-12T09:30:55.920096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:12.038992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":26427919,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":3194,"num_read_kb":30966,"num_write":187,"num_write_kb":16070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"92'218","reported_seq":7558,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.323049+0000","last_change":"2024-09-12T09:30:33.675916+0000","last_active":"2024-09-12T09:30:56.323049+0000","last_peered":"2024-09-12T09:30:56.323049+0000","last_clean":"2024-09-12T09:30:56.323049+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:30:56.323049+0000","last_undegraded":"2024-09-12T09:30:56.323049+0000","last_fullsized":"2024-09-12T09:30:56.323049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:48:34.796029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":23512703,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":8211,"num_read_kb":25480,"num_write":284,"num_write_kb":2777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"91'214","reported_seq":4708,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.661504+0000","last_change":"2024-09-12T09:30:27.834395+0000","last_active":"2024-09-12T09:30:55.661504+0000","last_peered":"2024-09-12T09:30:55.661504+0000","last_clean":"2024-09-12T09:30:55.661504+0000","last_became_active":"2024-09-12T09:30:27.833222+0000","last_became_peered":"2024-09-12T09:30:27.833222+0000","last_unstale":"2024-09-12T09:30:55.661504+0000","last_undegraded":"2024-09-12T09:30:55.661504+0000","last_fullsized":"2024-09-12T09:30:55.661504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:59:16.647260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":61129451,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":5302,"num_read_kb":90550,"num_write":253,"num_write_kb":44241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":55394924,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"92'291","reported_seq":5570,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.898059+0000","last_change":"2024-09-12T09:30:33.671011+0000","last_active":"2024-09-12T09:30:56.898059+0000","last_peered":"2024-09-12T09:30:56.898059+0000","last_clean":"2024-09-12T09:30:56.898059+0000","last_became_active":"2024-09-12T09:30:33.669882+0000","last_became_peered":"2024-09-12T09:30:33.669882+0000","last_unstale":"2024-09-12T09:30:56.898059+0000","last_undegraded":"2024-09-12T09:30:56.898059+0000","last_fullsized":"2024-09-12T09:30:56.898059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:15.274870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":84862663,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":5721,"num_read_kb":75025,"num_write":380,"num_write_kb":42716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":71454664,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'211","reported_seq":1259,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.887606+0000","last_change":"2024-09-12T09:30:03.657130+0000","last_active":"2024-09-12T09:30:55.887606+0000","last_peered":"2024-09-12T09:30:55.887606+0000","last_clean":"2024-09-12T09:30:55.887606+0000","last_became_active":"2024-09-12T09:30:03.655757+0000","last_became_peered":"2024-09-12T09:30:03.655757+0000","last_unstale":"2024-09-12T09:30:55.887606+0000","last_undegraded":"2024-09-12T09:30:55.887606+0000","last_fullsized":"2024-09-12T09:30:55.887606+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:30.707040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":36988298,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":1144,"num_read_kb":11046,"num_write":235,"num_write_kb":22817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":4010639,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"92'184","reported_seq":780,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.222428+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:30:56.222428+0000","last_peered":"2024-09-12T09:30:56.222428+0000","last_clean":"2024-09-12T09:30:56.222428+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:30:56.222428+0000","last_undegraded":"2024-09-12T09:30:56.222428+0000","last_fullsized":"2024-09-12T09:30:56.222428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:37.116513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":50520214,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":2520,"num_read_kb":10295,"num_write":204,"num_write_kb":1618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"91'232","reported_seq":10644,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T09:30:55.831797+0000","last_change":"2024-09-12T09:30:34.667415+0000","last_active":"2024-09-12T09:30:55.831797+0000","last_peered":"2024-09-12T09:30:55.831797+0000","last_clean":"2024-09-12T09:30:55.831797+0000","last_became_active":"2024-09-12T09:30:34.666472+0000","last_became_peered":"2024-09-12T09:30:34.666472+0000","last_unstale":"2024-09-12T09:30:55.831797+0000","last_undegraded":"2024-09-12T09:30:55.831797+0000","last_fullsized":"2024-09-12T09:30:55.831797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:06.326672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":71414865,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":11188,"num_read_kb":95224,"num_write":336,"num_write_kb":54728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":87166181,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"92'138","reported_seq":1261,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T09:30:56.880848+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:30:56.880848+0000","last_peered":"2024-09-12T09:30:56.880848+0000","last_clean":"2024-09-12T09:30:56.880848+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:30:56.880848+0000","last_undegraded":"2024-09-12T09:30:56.880848+0000","last_fullsized":"2024-09-12T09:30:56.880848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:37:57.527233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":37546675,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":1502,"num_read_kb":9141,"num_write":225,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1305475051,"num_objects":1304,"num_object_clones":305,"num_object_copies":2608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1304,"num_whiteouts":490,"num_read":171322,"num_read_kb":1205990,"num_write":10022,"num_write_kb":701657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":345852928,"num_keys_recovered":207,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73367552,"data_stored":1023456057,"data_compressed":10620926,"data_compressed_allocated":65101824,"data_compressed_original":1018009534,"omap_allocated":0,"internal_metadata":0},"log_size":7741,"ondisk_log_size":7741,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":2361472,"data_compressed":26616,"data_compressed_allocated":147456,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54672,"kb_used_data":15132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94317168,"statfs":{"total":96636764160,"available":96580780032,"internally_reserved":0,"allocated":15495168,"data_stored":200295491,"data_compressed":2142622,"data_compressed_allocated":12988416,"data_compressed_original":198805906,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51716,"kb_used_data":12176,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320124,"statfs":{"total":96636764160,"available":96583806976,"internally_reserved":0,"allocated":12468224,"data_stored":147219099,"data_compressed":1766494,"data_compressed_allocated":9883648,"data_compressed_original":145807769,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":60129542168,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54384,"kb_used_data":14868,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94317456,"statfs":{"total":96636764160,"available":96581074944,"internally_reserved":0,"allocated":15224832,"data_stored":216813346,"data_compressed":1653256,"data_compressed_allocated":12697600,"data_compressed_original":215267195,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38752,"kb_used_data":11708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333088,"statfs":{"total":96636764160,"available":96597082112,"internally_reserved":0,"allocated":11988992,"data_stored":148968330,"data_compressed":1761800,"data_compressed_allocated":9797632,"data_compressed_original":147831324,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43180,"kb_used_data":3676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328660,"statfs":{"total":96636764160,"available":96592547840,"internally_reserved":0,"allocated":3764224,"data_stored":27980990,"data_compressed":781627,"data_compressed_allocated":2183168,"data_compressed_original":27181701,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542167,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43240,"kb_used_data":16216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328600,"statfs":{"total":96636764160,"available":96592486400,"internally_reserved":0,"allocated":16605184,"data_stored":244306188,"data_compressed":1762337,"data_compressed_allocated":13905920,"data_compressed_original":242612257,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15507456,"data_stored":243267079,"data_compressed":1793849,"data_compressed_allocated":13897728,"data_compressed_original":242075681,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3252224,"data_stored":33159688,"data_compressed":430613,"data_compressed_allocated":2797568,"data_compressed_original":32897110,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2740224,"data_stored":27543057,"data_compressed":789404,"data_compressed_allocated":2195456,"data_compressed_original":27210373,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11026432,"data_stored":148689855,"data_compressed":1819411,"data_compressed_allocated":9879552,"data_compressed_original":148044316,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11726848,"data_stored":150362721,"data_compressed":1829241,"data_compressed_allocated":10186752,"data_compressed_original":149428633,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":14233600,"data_stored":216469983,"data_compressed":1684330,"data_compressed_allocated":12759040,"data_compressed_original":215410555,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":4709750,"data_compressed":109990,"data_compressed_allocated":405504,"data_compressed_original":4681728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14413824,"data_stored":199253924,"data_compressed":2164088,"data_compressed_allocated":12980224,"data_compressed_original":198261138,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:31:00.863 INFO:tasks.thrashosds.thrasher:Setting 3.1a to [0, 3, 4, 2] 2024-09-12T09:31:00.863 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1a', '0', '3', '4', '2'] 2024-09-12T09:31:00.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1a 0 3 4 2 2024-09-12T09:31:00.865 INFO:teuthology.orchestra.run.smithi007.stdout:3129 read 0x1606fdc thru 0x160ecf9 (0x7d1e bytes) 2024-09-12T09:31:00.865 INFO:teuthology.orchestra.run.smithi007.stdout:3130 write 0xa1f1a0d thru 0xa1fddd5 (0xc3c9 bytes) 2024-09-12T09:31:00.880 INFO:teuthology.orchestra.run.smithi007.stdout:3131 read 0xc7219ef thru 0xc72b0db (0x96ed bytes) 2024-09-12T09:31:00.880 INFO:teuthology.orchestra.run.smithi007.stdout:3132 read 0x5021ad7 thru 0x502633b (0x4865 bytes) 2024-09-12T09:31:00.880 INFO:teuthology.orchestra.run.smithi007.stdout:3134 write 0xa7bc858 thru 0xa7c81fc (0xb9a5 bytes) 2024-09-12T09:31:00.898 INFO:teuthology.orchestra.run.smithi007.stdout:3135 writesame 0xac0eded thru 0xac12e7e data_size 0x5f(0x4092 bytes) 2024-09-12T09:31:00.912 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.912 INFO:teuthology.orchestra.run.smithi007.stdout:3138 read 0xb526a6 thru 0xb5e1f9 (0xbb54 bytes) 2024-09-12T09:31:00.912 INFO:teuthology.orchestra.run.smithi007.stdout:3139 punch from 0x8bb9490 to 0x8bbde8c, (0x49fc bytes) 2024-09-12T09:31:00.912 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.912 INFO:teuthology.orchestra.run.smithi007.stdout:3142 writesame 0x6295623 thru 0x6297bd5 data_size 0x1(0x25b3 bytes) 2024-09-12T09:31:00.926 INFO:teuthology.orchestra.run.smithi007.stdout:3143 punch from 0x1b69df0 to 0x1b6a7e2, (0x9f2 bytes) 2024-09-12T09:31:00.928 INFO:teuthology.orchestra.run.smithi007.stdout:3145 trunc from 0xea553da to 0xcf4cca4 2024-09-12T09:31:00.941 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:00.940+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:00.956 INFO:teuthology.orchestra.run.smithi007.stdout:3146 punch from 0x4ae4c15 to 0x4af468a, (0xfa75 bytes) 2024-09-12T09:31:00.956 INFO:teuthology.orchestra.run.smithi007.stdout:3147 trunc from 0xcf4cca4 to 0xc9105df 2024-09-12T09:31:00.962 INFO:teuthology.orchestra.run.smithi007.stdout:3148 punch from 0x5a2a282 to 0x5a33d98, (0x9b16 bytes) 2024-09-12T09:31:00.962 INFO:teuthology.orchestra.run.smithi007.stdout:3150 write 0x77b3720 thru 0x77b4611 (0xef2 bytes) 2024-09-12T09:31:00.972 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.972 INFO:teuthology.orchestra.run.smithi007.stdout:3152 read 0x8f5154f thru 0x8f5aefb (0x99ad bytes) 2024-09-12T09:31:00.973 INFO:teuthology.orchestra.run.smithi007.stdout:3154 read 0x4ecb899 thru 0x4ed954f (0xdcb7 bytes) 2024-09-12T09:31:00.973 INFO:teuthology.orchestra.run.smithi007.stdout:3155 write 0x6011db thru 0x60c1d2 (0xaff8 bytes) 2024-09-12T09:31:00.977 INFO:teuthology.orchestra.run.smithi007.stdout:3156 write 0xac29fe7 thru 0xac37423 (0xd43d bytes) 2024-09-12T09:31:00.982 INFO:teuthology.orchestra.run.smithi007.stdout:3158 write 0xab83ddb thru 0xab93069 (0xf28f bytes) 2024-09-12T09:31:00.987 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:00.987 INFO:teuthology.orchestra.run.smithi007.stdout:3161 write 0x95a1c7c thru 0x95a78e2 (0x5c67 bytes) 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.995 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.856831024000002, 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:00.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:00.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:31:01.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:01.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755495482000001, 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:31:01.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:31:01.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755470029999998, 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:31:01.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:31:01.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:31:01.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:01.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755445178000002, 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:31:01.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755419840000002, 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:31:01.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755388809999999, 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:31:01.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:01.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755363649000003, 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:31:01.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755337748999999, 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:31:01.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755314143, 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:31:01.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:31:01.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755289630999997, 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:31:01.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755265193, 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:31:01.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:01.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755240606000001, 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:31:01.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755216590000003, 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:31:01.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755188748000002, 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:31:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755163903000003, 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:31:01.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:31:01.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755139176999997, 2024-09-12T09:31:01.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:01.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:31:01.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.755114679999998, 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:31:01.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.749308808999999, 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:31:01.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.604356844999998, 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:31:01.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:31:01.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:31:01.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:31:01.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.786860046999998, 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:31:01.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:31:01.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:01.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.102 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:01.103 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.103 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.103 INFO:teuthology.orchestra.run.smithi007.stdout:3164 writesame 0x36d8716 thru 0x36db880 data_size 0x1(0x316b bytes) 2024-09-12T09:31:01.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3165 read 0x11481b5 thru 0x1149a94 (0x18e0 bytes) 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3166 read 0x9787094 thru 0x9790dae (0x9d1b bytes) 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3167 read 0x56d0d82 thru 0x56de7ca (0xda49 bytes) 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3168 trunc from 0xc9105df to 0x7094f33 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3169 read 0x620e342 thru 0x621b738 (0xd3f7 bytes) 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3170 punch from 0x3434e47 to 0x3435201, (0x3ba bytes) 2024-09-12T09:31:01.105 INFO:teuthology.orchestra.run.smithi007.stdout:3171 trunc from 0x7094f33 to 0xa4b2567 2024-09-12T09:31:01.106 INFO:teuthology.orchestra.run.smithi007.stdout:3172 punch from 0x4180864 to 0x418fe95, (0xf631 bytes) 2024-09-12T09:31:01.106 INFO:teuthology.orchestra.run.smithi007.stdout:3176 trunc from 0xa4b2567 to 0x8e6e03 2024-09-12T09:31:01.106 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:01.099+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:01.113 INFO:teuthology.orchestra.run.smithi007.stdout:3179 read 0x4253c thru 0x48b04 (0x65c9 bytes) 2024-09-12T09:31:01.124 INFO:teuthology.orchestra.run.smithi007.stdout:3180 punch from 0x3e4128 to 0x3eb53a, (0x7412 bytes) 2024-09-12T09:31:01.133 INFO:teuthology.orchestra.run.smithi007.stdout:3181 write 0x1f8cf57 thru 0x1f9c662 (0xf70c bytes) 2024-09-12T09:31:01.155 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.169 INFO:teuthology.orchestra.run.smithi007.stdout:3184 write 0x74dc41a thru 0x74de12f (0x1d16 bytes) 2024-09-12T09:31:01.181 INFO:teuthology.orchestra.run.smithi007.stdout:3185 write 0x195ba71 thru 0x195f7fa (0x3d8a bytes) 2024-09-12T09:31:01.201 INFO:teuthology.orchestra.run.smithi007.stdout:3186 read 0x6030b9a thru 0x603d7e9 (0xcc50 bytes) 2024-09-12T09:31:01.201 INFO:teuthology.orchestra.run.smithi007.stdout:3192 writesame 0xa48559 thru 0xa4907a data_size 0x1e(0xb22 bytes) 2024-09-12T09:31:01.204 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:01.203+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:01.211 INFO:teuthology.orchestra.run.smithi007.stdout:3193 write 0x5717520 thru 0x571a2e3 (0x2dc4 bytes) 2024-09-12T09:31:01.237 INFO:teuthology.orchestra.run.smithi007.stdout:3194 writesame 0xafbf076 thru 0xafbfaa9 data_size 0x2(0xa34 bytes) 2024-09-12T09:31:01.251 INFO:teuthology.orchestra.run.smithi007.stdout:3195 trunc from 0xafbfaaa to 0x60f0b9 2024-09-12T09:31:01.252 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:01.253 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:01.253 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:01.253 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:01.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:01.305 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:01.303+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:01.309 INFO:teuthology.orchestra.run.smithi007.stdout:3196 write 0x7402f3a thru 0x7411b49 (0xec10 bytes) 2024-09-12T09:31:01.327 INFO:teuthology.orchestra.run.smithi007.stdout:3198 read 0xca9a1f thru 0xcb69fe (0xcfe0 bytes) 2024-09-12T09:31:01.339 INFO:teuthology.orchestra.run.smithi007.stdout:3199 write 0xb244161 thru 0xb24bf3e (0x7dde bytes) 2024-09-12T09:31:01.356 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.356 INFO:teuthology.orchestra.run.smithi007.stdout:3201 read 0xa3876fe thru 0xa388219 (0xb1c bytes) 2024-09-12T09:31:01.356 INFO:teuthology.orchestra.run.smithi007.stdout:3202 writesame 0x7f7f00 thru 0x801530 data_size 0x1(0x9631 bytes) 2024-09-12T09:31:01.378 INFO:teuthology.orchestra.run.smithi007.stdout:3203 read 0x84aa791 thru 0x84b3b1f (0x938f bytes) 2024-09-12T09:31:01.378 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.378 INFO:teuthology.orchestra.run.smithi007.stdout:3206 read 0x714238f thru 0x714805c (0x5cce bytes) 2024-09-12T09:31:01.379 INFO:teuthology.orchestra.run.smithi007.stdout:3208 punch from 0x6781ae1 to 0x6783d4b, (0x226a bytes) 2024-09-12T09:31:01.388 INFO:teuthology.orchestra.run.smithi007.stdout:3212 write 0xcf85f13 thru 0xcf86e4b (0xf39 bytes) 2024-09-12T09:31:01.398 INFO:teuthology.orchestra.run.smithi007.stdout:3214 read 0x5ef5562 thru 0x5ef6fd2 (0x1a71 bytes) 2024-09-12T09:31:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:3217 punch from 0x45ac43 to 0x465e11, (0xb1ce bytes) 2024-09-12T09:31:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:3218 clone 34 order 24 su 4194304 sc 12 2024-09-12T09:31:01.406 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:01.404+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:01.436 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:01.436 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:01.436 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:01.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:01.507 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:01.503+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:01.603 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.402277220000002, 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:01.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:31:01.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.387306490999997, 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:31:01.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:01.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070459487000001, 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:31:01.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:01.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070413127999998, 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:31:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070305908000002, 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:31:01.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:31:01.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070212417, 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:31:01.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070145904, 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:01.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:31:01.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070085095, 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:31:01.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.070025803, 2024-09-12T09:31:01.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:01.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:31:01.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069965293999999, 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:31:01.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069887239, 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:31:01.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:31:01.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069801863999999, 2024-09-12T09:31:01.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:31:01.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:31:01.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069733544999998, 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:31:01.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:31:01.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069640960000001, 2024-09-12T09:31:01.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:31:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069585051000001, 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:31:01.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069552728000001, 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:31:01.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:31:01.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.680 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069519135, 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:31:01.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069468192999999, 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:31:01.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069370037999999, 2024-09-12T09:31:01.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:31:01.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.069312944, 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:01.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:31:01.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:01.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:01.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:01.699 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:01.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:01.701 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:01.699+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:01.801 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:01.800+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:01.830 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:01.830 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:01.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:01.902 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:01.899+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:02.003 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:01.999+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:02.013 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:02.013 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:02.013 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:02.013 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:02.013 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:02.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:02.104 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:02.104+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1a pg_upmap_items mapping to [0->3,4->2] 2024-09-12T09:31:02.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.853409992000003, 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.670991074, 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.278204561000003, 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.930674672999999, 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.928220762999999, 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:02.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.179 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.926110876000003, 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:02.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.430364231, 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:02.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:02.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.399955945, 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:02.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:02.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1789913859999999, 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:02.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:02.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:02.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4798801029999999, 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:02.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4798631740000001, 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:02.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:02.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:02.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.217 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4411015759999999, 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.218 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:02.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:02.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.383733087, 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:02.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:02.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.3837096680000001, 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:02.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:02.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.3524806630000001, 2024-09-12T09:31:02.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:02.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:02.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.271377706, 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:02.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:02.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.2713517109999999, 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:02.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:02.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.018135223, 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:02.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:02.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:02.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.79545114699999997, 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:02.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:02.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.79542433899999998, 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:02.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:02.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:02.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:02.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:02.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:02.275 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:02.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:02.277 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone33 from 0xcf86e4c (overlap 0x60f0b9) to 0x573f0c 2024-09-12T09:31:02.278 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:02.276+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:02.378 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:02.375+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:02.430 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-12T09:31:02.438 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:02.438 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:02.438 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:02.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:02.496 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:02.496+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:02.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:02.574 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:02.575 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:02.579 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:02.575+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:02.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:02.684 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:02.684+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:02.691 INFO:teuthology.orchestra.run.smithi007.stdout:3220 read 0xb1b8d11 thru 0xb1be01f (0x530f bytes) 2024-09-12T09:31:02.693 INFO:teuthology.orchestra.run.smithi007.stdout:3221 read 0x8139a9c thru 0x813df0d (0x4472 bytes) 2024-09-12T09:31:02.710 INFO:teuthology.orchestra.run.smithi007.stdout:3222 writesame 0xd78ad7b thru 0xd78c209 data_size 0x1(0x148f bytes) 2024-09-12T09:31:02.711 INFO:teuthology.orchestra.run.smithi007.stdout:3223 write 0xa0fa680 thru 0xa0fe96c (0x42ed bytes) 2024-09-12T09:31:02.719 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:02.720 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:02.721 INFO:teuthology.orchestra.run.smithi007.stdout:3224 read 0x641e54c thru 0x6423fd0 (0x5a85 bytes) 2024-09-12T09:31:02.721 INFO:teuthology.orchestra.run.smithi007.stdout:3225 punch from 0xa5678e3 to 0xa572507, (0xac24 bytes) 2024-09-12T09:31:02.721 INFO:teuthology.orchestra.run.smithi007.stdout:3227 punch from 0xac7cdc2 to 0xac7d22b, (0x469 bytes) 2024-09-12T09:31:02.721 INFO:teuthology.orchestra.run.smithi007.stdout:3228 write 0x8b082e1 thru 0x8b096bb (0x13db bytes) 2024-09-12T09:31:02.731 INFO:teuthology.orchestra.run.smithi007.stdout:3231 punch from 0x30d2f23 to 0x30d3f5c, (0x1039 bytes) 2024-09-12T09:31:02.732 INFO:teuthology.orchestra.run.smithi007.stdout:3232 punch from 0x3259e33 to 0x32633a3, (0x9570 bytes) 2024-09-12T09:31:02.732 INFO:teuthology.orchestra.run.smithi007.stdout:3235 write 0x3416c3 thru 0x350361 (0xec9f bytes) 2024-09-12T09:31:02.742 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:02.742 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:02.770 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:02.781 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:02.780+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:02.802 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:02.802 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:02.802 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:02.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:02.841 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:02.841 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:02.876 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:02.881 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:02.880+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:02.903 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:02.927 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:02.983 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:02.979+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:03.000 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.000 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:03.100 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:03.099+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:31:03.158 INFO:teuthology.orchestra.run.smithi007.stdout:3236 read 0x5d43e12 thru 0x5d4f83a (0xba29 bytes) 2024-09-12T09:31:03.159 INFO:teuthology.orchestra.run.smithi007.stdout:3237 read 0x851544b thru 0x851e290 (0x8e46 bytes) 2024-09-12T09:31:03.159 INFO:teuthology.orchestra.run.smithi007.stdout:3238 punch from 0x763b125 to 0x763bc29, (0xb04 bytes) 2024-09-12T09:31:03.159 INFO:teuthology.orchestra.run.smithi007.stdout:3239 write 0x771858 thru 0x77c131 (0xa8da bytes) 2024-09-12T09:31:03.184 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:03.183+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:03.190 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.191 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:03.286 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:03.284+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:03.302 INFO:teuthology.orchestra.run.smithi007.stdout:3240 trunc from 0xd78c20a to 0x9e1487c 2024-09-12T09:31:03.353 INFO:teuthology.orchestra.run.smithi007.stdout:3241 write 0x431bf7c thru 0x432bef4 (0xff79 bytes) 2024-09-12T09:31:03.355 INFO:teuthology.orchestra.run.smithi007.stdout:3242 writesame 0x499f137 thru 0x499f7e0 data_size 0x1(0x6aa bytes) 2024-09-12T09:31:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:3243 read 0x7c6d9dc thru 0x7c76ee1 (0x9506 bytes) 2024-09-12T09:31:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:3244 read 0x6826001 thru 0x68339f2 (0xd9f2 bytes) 2024-09-12T09:31:03.370 INFO:teuthology.orchestra.run.smithi007.stdout:3246 writesame 0xa4fdc9b thru 0xa5041c3 data_size 0x1(0x6529 bytes) 2024-09-12T09:31:03.372 INFO:teuthology.orchestra.run.smithi007.stdout:3247 writesame 0x607c339 thru 0x6082b4e data_size 0x3(0x6816 bytes) 2024-09-12T09:31:03.374 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.374 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.386 INFO:teuthology.orchestra.run.smithi007.stdout:3248 trunc from 0xa5041c4 to 0x74adb76 2024-09-12T09:31:03.387 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:03.384+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:03.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:03.412 INFO:teuthology.orchestra.run.smithi007.stdout:3249 read 0x2d29df8 thru 0x2d2ef69 (0x5172 bytes) 2024-09-12T09:31:03.412 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.416 INFO:teuthology.orchestra.run.smithi007.stdout:3252 trunc from 0x74adb76 to 0x9678529 2024-09-12T09:31:03.433 INFO:teuthology.orchestra.run.smithi007.stdout:3253 trunc from 0x9678529 to 0xc5514fd 2024-09-12T09:31:03.444 INFO:teuthology.orchestra.run.smithi007.stdout:3254 punch from 0x6080f50 to 0x6087978, (0x6a28 bytes) 2024-09-12T09:31:03.445 INFO:teuthology.orchestra.run.smithi007.stdout:3255 writesame 0x1851018 thru 0x185dd4c data_size 0x27(0xcd35 bytes) 2024-09-12T09:31:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:3256 writesame 0x8274e0c thru 0x827637a data_size 0x1f(0x156f bytes) 2024-09-12T09:31:03.451 INFO:teuthology.orchestra.run.smithi007.stdout:3257 write 0x4288b4b thru 0x4289c11 (0x10c7 bytes) 2024-09-12T09:31:03.454 INFO:teuthology.orchestra.run.smithi007.stdout:3258 read 0xa098920 thru 0xa0a809b (0xf77c bytes) 2024-09-12T09:31:03.475 INFO:teuthology.orchestra.run.smithi007.stdout:3259 writesame 0xed04204 thru 0xed0f448 data_size 0x1(0xb245 bytes) 2024-09-12T09:31:03.475 INFO:teuthology.orchestra.run.smithi007.stdout:3260 write 0x5913b8c thru 0x5916293 (0x2708 bytes) 2024-09-12T09:31:03.478 INFO:teuthology.orchestra.run.smithi007.stdout:3261 write 0x44ab79c thru 0x44ba63d (0xeea2 bytes) 2024-09-12T09:31:03.481 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.481 INFO:teuthology.orchestra.run.smithi007.stdout:3263 read 0x6cb9017 thru 0x6cc0c04 (0x7bee bytes) 2024-09-12T09:31:03.482 INFO:teuthology.orchestra.run.smithi007.stdout:3264 read 0xbf3a10b thru 0xbf43e80 (0x9d76 bytes) 2024-09-12T09:31:03.483 INFO:teuthology.orchestra.run.smithi007.stdout:3265 read 0x2c65bd4 thru 0x2c665e1 (0xa0e bytes) 2024-09-12T09:31:03.484 INFO:teuthology.orchestra.run.smithi007.stdout:3266 writesame 0x183544c thru 0x1844630 data_size 0x5(0xf1e5 bytes) 2024-09-12T09:31:03.488 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:03.484+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:03.494 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.494 INFO:teuthology.orchestra.run.smithi007.stdout:3269 write 0xb93a072 thru 0xb93a5f0 (0x57f bytes) 2024-09-12T09:31:03.497 INFO:teuthology.orchestra.run.smithi007.stdout:3270 read 0xac20134 thru 0xac2e1ff (0xe0cc bytes) 2024-09-12T09:31:03.499 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.499 INFO:teuthology.orchestra.run.smithi007.stdout:3274 punch from 0xb4eaa8 to 0xb588e5, (0x9e3d bytes) 2024-09-12T09:31:03.499 INFO:teuthology.orchestra.run.smithi007.stdout:3275 punch from 0x1b0910 to 0x1bb99e, (0xb08e bytes) 2024-09-12T09:31:03.499 INFO:teuthology.orchestra.run.smithi007.stdout:3276 trunc from 0xed0f449 to 0xeca7c4d 2024-09-12T09:31:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:3279 write 0xde5ee90 thru 0xde6e7e7 (0xf958 bytes) 2024-09-12T09:31:03.514 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.514 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:03.551 INFO:teuthology.orchestra.run.smithi007.stdout:3282 trunc from 0xeca7c4d to 0x201e370 2024-09-12T09:31:03.580 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.588 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:03.588+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:03.612 INFO:teuthology.orchestra.run.smithi007.stdout:3284 write 0xa846502 thru 0xa84b7ed (0x52ec bytes) 2024-09-12T09:31:03.616 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.631 INFO:teuthology.orchestra.run.smithi007.stdout:3286 write 0xbf51bf2 thru 0xbf5e623 (0xca32 bytes) 2024-09-12T09:31:03.638 INFO:teuthology.orchestra.run.smithi007.stdout:3287 write 0x5d1c83a thru 0x5d21786 (0x4f4d bytes) 2024-09-12T09:31:03.642 INFO:teuthology.orchestra.run.smithi007.stdout:3289 read 0x6704736 thru 0x670dcce (0x9599 bytes) 2024-09-12T09:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout:3290 punch from 0xb86ba54 to 0xb86d9d8, (0x1f84 bytes) 2024-09-12T09:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout:3291 read 0x26b02ba thru 0x26b94b4 (0x91fb bytes) 2024-09-12T09:31:03.643 INFO:teuthology.orchestra.run.smithi007.stdout:3292 write 0x4349928 thru 0x434e27a (0x4953 bytes) 2024-09-12T09:31:03.647 INFO:teuthology.orchestra.run.smithi007.stdout:3293 write 0xd87c57 thru 0xd9189e (0x9c48 bytes) 2024-09-12T09:31:03.656 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.656 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.665 INFO:teuthology.orchestra.run.smithi007.stdout:3294 trunc from 0xbf5e624 to 0x6e01008 2024-09-12T09:31:03.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:03.694 INFO:teuthology.orchestra.run.smithi007.stdout:3295 trunc from 0x6e01008 to 0xb54f768 2024-09-12T09:31:03.705 INFO:teuthology.orchestra.run.smithi007.stdout:3296 writesame 0xc36e07 thru 0xc3d62a data_size 0x2b(0x6824 bytes) 2024-09-12T09:31:03.708 INFO:teuthology.orchestra.run.smithi007.stdout:3297 writesame 0x939cc8c thru 0x93a8b8c data_size 0x9(0xbf01 bytes) 2024-09-12T09:31:03.716 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:03.715+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:03.724 INFO:teuthology.orchestra.run.smithi007.stdout:3298 writesame 0xb8b7053 thru 0xb8c1d60 data_size 0x11(0xad0e bytes) 2024-09-12T09:31:03.724 INFO:teuthology.orchestra.run.smithi007.stdout:3300 punch from 0x201aed9 to 0x2024746, (0x986d bytes) 2024-09-12T09:31:03.724 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.724 INFO:teuthology.orchestra.run.smithi007.stdout:3302 read 0xb6da849 thru 0xb6e5f85 (0xb73d bytes) 2024-09-12T09:31:03.742 INFO:teuthology.orchestra.run.smithi007.stdout:3303 write 0xd89180e thru 0xd8928ed (0x10e0 bytes) 2024-09-12T09:31:03.747 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.747 INFO:teuthology.orchestra.run.smithi007.stdout:3307 writesame 0x716b788 thru 0x7177580 data_size 0x57(0xbdf9 bytes) 2024-09-12T09:31:03.751 INFO:teuthology.orchestra.run.smithi007.stdout:3309 punch from 0xc0e0c4f to 0xc0e8ddc, (0x818d bytes) 2024-09-12T09:31:03.751 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.751 INFO:teuthology.orchestra.run.smithi007.stdout:3312 writesame 0xb38e148 thru 0xb38e168 data_size 0x1(0x21 bytes) 2024-09-12T09:31:03.754 INFO:teuthology.orchestra.run.smithi007.stdout:3313 write 0x14427f4 thru 0x1446af0 (0x42fd bytes) 2024-09-12T09:31:03.758 INFO:teuthology.orchestra.run.smithi007.stdout:3314 writesame 0xaac74b1 thru 0xaad38a1 data_size 0x1(0xc3f1 bytes) 2024-09-12T09:31:03.790 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:03.787+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:03.798 INFO:teuthology.orchestra.run.smithi007.stdout:3316 write 0xc514498 thru 0xc5233f7 (0xef60 bytes) 2024-09-12T09:31:03.802 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.802 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.807 INFO:teuthology.orchestra.run.smithi007.stdout:3317 read 0x263fe95 thru 0x2647a5d (0x7bc9 bytes) 2024-09-12T09:31:03.807 INFO:teuthology.orchestra.run.smithi007.stdout:3319 write 0x33c4f thru 0x42b40 (0xeef2 bytes) 2024-09-12T09:31:03.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:03.827 INFO:teuthology.orchestra.run.smithi007.stdout:3322 write 0xe35e5d3 thru 0xe36097e (0x23ac bytes) 2024-09-12T09:31:03.835 INFO:teuthology.orchestra.run.smithi007.stdout:3323 read 0xbfc4533 thru 0xbfc7189 (0x2c57 bytes) 2024-09-12T09:31:03.864 INFO:teuthology.orchestra.run.smithi007.stdout:3325 trunc from 0xe36097f to 0x3eb5f19 2024-09-12T09:31:03.894 INFO:teuthology.orchestra.run.smithi007.stdout:3326 read 0x260b4fc thru 0x260cbd4 (0x16d9 bytes) 2024-09-12T09:31:03.906 INFO:teuthology.orchestra.run.smithi007.stdout:3328 trunc from 0x3eb5f19 to 0x83b08eb 2024-09-12T09:31:03.908 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:03.907+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:03.919 INFO:teuthology.orchestra.run.smithi007.stdout:3329 write 0xabb0b2 thru 0xac1d29 (0x6c78 bytes) 2024-09-12T09:31:03.950 INFO:teuthology.orchestra.run.smithi007.stdout:3330 writesame 0xe0bd73b thru 0xe0c1c3c data_size 0x16(0x4502 bytes) 2024-09-12T09:31:03.950 INFO:teuthology.orchestra.run.smithi007.stdout:3331 read 0xbf46a8d thru 0xbf5437f (0xd8f3 bytes) 2024-09-12T09:31:03.951 INFO:teuthology.orchestra.run.smithi007.stdout:3333 write 0x6d8b83d thru 0x6d9a74b (0xef0f bytes) 2024-09-12T09:31:03.955 INFO:teuthology.orchestra.run.smithi007.stdout:3334 write 0x2fb67ab thru 0x2fc35b6 (0xce0c bytes) 2024-09-12T09:31:03.959 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.960 INFO:teuthology.orchestra.run.smithi007.stdout:3336 write 0x12f5a98 thru 0x12ff51f (0x9a88 bytes) 2024-09-12T09:31:03.963 INFO:teuthology.orchestra.run.smithi007.stdout:3339 read 0x5a38ce6 thru 0x5a45f55 (0xd270 bytes) 2024-09-12T09:31:03.963 INFO:teuthology.orchestra.run.smithi007.stdout:3340 punch from 0xad7064c to 0xad7a76a, (0xa11e bytes) 2024-09-12T09:31:03.963 INFO:teuthology.orchestra.run.smithi007.stdout:3341 read 0x48bdb92 thru 0x48ccfb6 (0xf425 bytes) 2024-09-12T09:31:03.964 INFO:teuthology.orchestra.run.smithi007.stdout:3342 punch from 0x4685c52 to 0x46885f3, (0x29a1 bytes) 2024-09-12T09:31:03.964 INFO:teuthology.orchestra.run.smithi007.stdout:3343 write 0xc1f5a51 thru 0xc1fd088 (0x7638 bytes) 2024-09-12T09:31:03.973 INFO:teuthology.orchestra.run.smithi007.stdout:3345 write 0x55b2620 thru 0x55b8527 (0x5f08 bytes) 2024-09-12T09:31:03.979 INFO:teuthology.orchestra.run.smithi007.stdout:3348 read 0xbead179 thru 0xbeafaa9 (0x2931 bytes) 2024-09-12T09:31:03.980 INFO:teuthology.orchestra.run.smithi007.stdout:3349 read 0xcf34f01 thru 0xcf41190 (0xc290 bytes) 2024-09-12T09:31:03.980 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:03.992 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:03.992+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:03.994 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:03.994 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:03.997 INFO:teuthology.orchestra.run.smithi007.stdout:3351 trunc from 0xe0c1c3d to 0x7c81edb 2024-09-12T09:31:04.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:04.047 INFO:teuthology.orchestra.run.smithi007.stdout:3352 write 0xaf47e64 thru 0xaf4ba08 (0x3ba5 bytes) 2024-09-12T09:31:04.068 INFO:teuthology.orchestra.run.smithi007.stdout:3353 write 0xc353faa thru 0xc35925c (0x52b3 bytes) 2024-09-12T09:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.074 INFO:teuthology.orchestra.run.smithi007.stdout:3356 read 0xa0f0eb5 thru 0xa0f5530 (0x467c bytes) 2024-09-12T09:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout:3358 punch from 0xa222764 to 0xa2300cf, (0xd96b bytes) 2024-09-12T09:31:04.075 INFO:teuthology.orchestra.run.smithi007.stdout:3359 read 0x7f02e91 thru 0x7f08c92 (0x5e02 bytes) 2024-09-12T09:31:04.076 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.077 INFO:teuthology.orchestra.run.smithi007.stdout:3362 writesame 0x656e83a thru 0x6574ebd data_size 0x6c(0x6684 bytes) 2024-09-12T09:31:04.079 INFO:teuthology.orchestra.run.smithi007.stdout:3363 write 0xa9c2a2c thru 0xa9c89a7 (0x5f7c bytes) 2024-09-12T09:31:04.083 INFO:teuthology.orchestra.run.smithi007.stdout:3366 read 0x636cf84 thru 0x6374679 (0x76f6 bytes) 2024-09-12T09:31:04.084 INFO:teuthology.orchestra.run.smithi007.stdout:3368 writesame 0xab2151c thru 0xab26037 data_size 0x17(0x4b1c bytes) 2024-09-12T09:31:04.087 INFO:teuthology.orchestra.run.smithi007.stdout:3369 punch from 0x7a73c6a to 0x7a7479b, (0xb31 bytes) 2024-09-12T09:31:04.088 INFO:teuthology.orchestra.run.smithi007.stdout:3370 read 0x8cfbc0e thru 0x8cfe5e0 (0x29d3 bytes) 2024-09-12T09:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout:3371 read 0x7b611c1 thru 0x7b6e773 (0xd5b3 bytes) 2024-09-12T09:31:04.089 INFO:teuthology.orchestra.run.smithi007.stdout:3373 punch from 0x7c307b5 to 0x7c325a8, (0x1df3 bytes) 2024-09-12T09:31:04.096 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:04.095+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:04.108 INFO:teuthology.orchestra.run.smithi007.stdout:3374 trunc from 0xc35925d to 0x57aa618 2024-09-12T09:31:04.137 INFO:teuthology.orchestra.run.smithi007.stdout:3377 trunc from 0x57aa618 to 0x56d4b74 2024-09-12T09:31:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:3380 read 0x27a059 thru 0x286b22 (0xcaca bytes) 2024-09-12T09:31:04.154 INFO:teuthology.orchestra.run.smithi007.stdout:3381 trunc from 0x56d4b74 to 0x49a23db 2024-09-12T09:31:04.184 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:04.185 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:04.188 INFO:teuthology.orchestra.run.smithi007.stdout:3382 writesame 0x5981312 thru 0x598e463 data_size 0x27(0xd152 bytes) 2024-09-12T09:31:04.194 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:04.192+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:04.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:04.207 INFO:teuthology.orchestra.run.smithi007.stdout:3383 trunc from 0x598e464 to 0xced4707 2024-09-12T09:31:04.231 INFO:teuthology.orchestra.run.smithi007.stdout:3384 write 0xd29682f thru 0xd29ba3d (0x520f bytes) 2024-09-12T09:31:04.237 INFO:teuthology.orchestra.run.smithi007.stdout:3385 writesame 0x209d9b3 thru 0x20a91da data_size 0x8(0xb828 bytes) 2024-09-12T09:31:04.253 INFO:teuthology.orchestra.run.smithi007.stdout:3386 trunc from 0xd29ba3e to 0x9d461ba 2024-09-12T09:31:04.289 INFO:teuthology.orchestra.run.smithi007.stdout:3387 writesame 0xa593f85 thru 0xa599e95 data_size 0x1(0x5f11 bytes) 2024-09-12T09:31:04.292 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.292 INFO:teuthology.orchestra.run.smithi007.stdout:3389 read 0x393dd20 thru 0x394b795 (0xda76 bytes) 2024-09-12T09:31:04.294 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:04.292+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:04.317 INFO:teuthology.orchestra.run.smithi007.stdout:3390 writesame 0xe2bc99f thru 0xe2bda28 data_size 0x1d(0x108a bytes) 2024-09-12T09:31:04.317 INFO:teuthology.orchestra.run.smithi007.stdout:3394 punch from 0xbf6f9cb to 0xbf7a122, (0xa757 bytes) 2024-09-12T09:31:04.317 INFO:teuthology.orchestra.run.smithi007.stdout:3395 write 0x737f20f thru 0x7384631 (0x5423 bytes) 2024-09-12T09:31:04.320 INFO:teuthology.orchestra.run.smithi007.stdout:3396 read 0x12f03b8 thru 0x12feef7 (0xeb40 bytes) 2024-09-12T09:31:04.332 INFO:teuthology.orchestra.run.smithi007.stdout:3397 trunc from 0xe2bda29 to 0xa2e9282 2024-09-12T09:31:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:3398 punch from 0x728d22c to 0x729ca08, (0xf7dc bytes) 2024-09-12T09:31:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:3400 read 0x9d04bb9 thru 0x9d0bf11 (0x7359 bytes) 2024-09-12T09:31:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.350 INFO:teuthology.orchestra.run.smithi007.stdout:3403 read 0x9a4748c thru 0x9a4997c (0x24f1 bytes) 2024-09-12T09:31:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:3404 punch from 0x499117c to 0x49953ff, (0x4283 bytes) 2024-09-12T09:31:04.351 INFO:teuthology.orchestra.run.smithi007.stdout:3405 read 0x3f50196 thru 0x3f580bb (0x7f26 bytes) 2024-09-12T09:31:04.369 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:04.370 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:3407 write 0xd1f63d7 thru 0xd1ff90b (0x9535 bytes) 2024-09-12T09:31:04.385 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:04.395 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:04.392+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:04.399 INFO:teuthology.orchestra.run.smithi007.stdout:3409 trunc from 0xd1ff90c to 0x5f7327b 2024-09-12T09:31:04.456 INFO:teuthology.orchestra.run.smithi007.stdout:3410 writesame 0xc11dc37 thru 0xc122453 data_size 0x1(0x481d bytes) 2024-09-12T09:31:04.456 INFO:teuthology.orchestra.run.smithi007.stdout:3411 writesame 0x5880046 thru 0x58816f3 data_size 0x1(0x16ae bytes) 2024-09-12T09:31:04.462 INFO:teuthology.orchestra.run.smithi007.stdout:3413 read 0x6178b68 thru 0x61802c8 (0x7761 bytes) 2024-09-12T09:31:04.463 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.463 INFO:teuthology.orchestra.run.smithi007.stdout:3415 read 0x920174 thru 0x9260b2 (0x5f3f bytes) 2024-09-12T09:31:04.476 INFO:teuthology.orchestra.run.smithi007.stdout:3418 write 0xc6c317c thru 0xc6c9e56 (0x6cdb bytes) 2024-09-12T09:31:04.486 INFO:teuthology.orchestra.run.smithi007.stdout:3420 trunc from 0xc6c9e57 to 0xd5cec88 2024-09-12T09:31:04.496 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:04.495+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:04.498 INFO:teuthology.orchestra.run.smithi007.stdout:3421 read 0x846cfa thru 0x8550b4 (0xe3bb bytes) 2024-09-12T09:31:04.499 INFO:teuthology.orchestra.run.smithi007.stdout:3422 write 0x2dbcced thru 0x2dc2e8e (0x61a2 bytes) 2024-09-12T09:31:04.503 INFO:teuthology.orchestra.run.smithi007.stdout:3423 read 0xcc1b47 thru 0xccff3c (0xe3f6 bytes) 2024-09-12T09:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout:3424 punch from 0x84616 to 0x8a56b, (0x5f55 bytes) 2024-09-12T09:31:04.504 INFO:teuthology.orchestra.run.smithi007.stdout:3425 write 0x85564e3 thru 0x85605e7 (0xa105 bytes) 2024-09-12T09:31:04.508 INFO:teuthology.orchestra.run.smithi007.stdout:3426 read 0xc782d thru 0xd6ced (0xf4c1 bytes) 2024-09-12T09:31:04.509 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:04.509 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.509 INFO:teuthology.orchestra.run.smithi007.stdout:3431 punch from 0xb260826 to 0xb262166, (0x1940 bytes) 2024-09-12T09:31:04.510 INFO:teuthology.orchestra.run.smithi007.stdout:3432 writesame 0x67d9004 thru 0x67e7b96 data_size 0x1(0xeb93 bytes) 2024-09-12T09:31:04.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:04.565 INFO:teuthology.orchestra.run.smithi007.stdout:3433 trunc from 0xd5cec88 to 0xe7d2c6b 2024-09-12T09:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout:3434 punch from 0x9a4d2b8 to 0x9a5a38b, (0xd0d3 bytes) 2024-09-12T09:31:04.577 INFO:teuthology.orchestra.run.smithi007.stdout:3436 write 0x84a829a thru 0x84abf0f (0x3c76 bytes) 2024-09-12T09:31:04.579 INFO:teuthology.orchestra.run.smithi007.stdout:3437 write 0xd328f4 thru 0xd3cbf0 (0xa2fd bytes) 2024-09-12T09:31:04.582 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.597 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:04.596+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:04.608 INFO:teuthology.orchestra.run.smithi007.stdout:3439 trunc from 0xe7d2c6b to 0x4ae936e 2024-09-12T09:31:04.653 INFO:teuthology.orchestra.run.smithi007.stdout:3440 writesame 0x729d43a thru 0x729eb2f data_size 0x1(0x16f6 bytes) 2024-09-12T09:31:04.655 INFO:teuthology.orchestra.run.smithi007.stdout:3441 read 0x21cbbfd thru 0x21d7777 (0xbb7b bytes) 2024-09-12T09:31:04.682 INFO:teuthology.orchestra.run.smithi007.stdout:3442 write 0xc4e1419 thru 0xc4e7118 (0x5d00 bytes) 2024-09-12T09:31:04.698 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:04.695+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:04.699 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:04.699 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:04.711 INFO:teuthology.orchestra.run.smithi007.stdout:3443 write 0xe0afddd thru 0xe0bfc49 (0xfe6d bytes) 2024-09-12T09:31:04.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:04.719 INFO:teuthology.orchestra.run.smithi007.stdout:3444 read 0xbacdc24 thru 0xbad8146 (0xa523 bytes) 2024-09-12T09:31:04.720 INFO:teuthology.orchestra.run.smithi007.stdout:3445 write 0x2db36ab thru 0x2dba092 (0x69e8 bytes) 2024-09-12T09:31:04.724 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.724 INFO:teuthology.orchestra.run.smithi007.stdout:3447 writesame 0xcb6f212 thru 0xcb6fe7a data_size 0x9(0xc69 bytes) 2024-09-12T09:31:04.731 INFO:teuthology.orchestra.run.smithi007.stdout:3449 read 0xdce2fb9 thru 0xdcecae6 (0x9b2e bytes) 2024-09-12T09:31:04.751 INFO:teuthology.orchestra.run.smithi007.stdout:3451 trunc from 0xe0bfc4a to 0x6c362b7 2024-09-12T09:31:04.782 INFO:teuthology.orchestra.run.smithi007.stdout:3452 punch from 0x260c5d7 to 0x2610d5b, (0x4784 bytes) 2024-09-12T09:31:04.783 INFO:teuthology.orchestra.run.smithi007.stdout:3454 read 0x49c890a thru 0x49d3362 (0xaa59 bytes) 2024-09-12T09:31:04.800 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:04.799+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:04.812 INFO:teuthology.orchestra.run.smithi007.stdout:3455 write 0xd53b064 thru 0xd53fbc8 (0x4b65 bytes) 2024-09-12T09:31:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:3459 write 0xa3b896c thru 0xa3c6cb6 (0xe34b bytes) 2024-09-12T09:31:04.824 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.825 INFO:teuthology.orchestra.run.smithi007.stdout:3462 writesame 0x13b3042 thru 0x13be2c4 data_size 0x1(0xb283 bytes) 2024-09-12T09:31:04.876 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:04.876 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:04.884 INFO:teuthology.orchestra.run.smithi007.stdout:3463 writesame 0xa4b6091 thru 0xa4b82e7 data_size 0x1(0x2257 bytes) 2024-09-12T09:31:04.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:04.892 INFO:teuthology.orchestra.run.smithi007.stdout:3464 read 0x169828c thru 0x169cefe (0x4c73 bytes) 2024-09-12T09:31:04.894 INFO:teuthology.orchestra.run.smithi007.stdout:3467 punch from 0xb93a69c to 0xb93bc9c, (0x1600 bytes) 2024-09-12T09:31:04.894 INFO:teuthology.orchestra.run.smithi007.stdout:3468 writesame 0x5380fc8 thru 0x538d669 data_size 0x96(0xc6a2 bytes) 2024-09-12T09:31:04.898 INFO:teuthology.orchestra.run.smithi007.stdout:3469 writesame 0xcb2a263 thru 0xcb33b3a data_size 0x49(0x98d8 bytes) 2024-09-12T09:31:04.905 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.905 INFO:teuthology.orchestra.run.smithi007.stdout:3471 write 0x88733ef thru 0x8874776 (0x1388 bytes) 2024-09-12T09:31:04.907 INFO:teuthology.orchestra.run.smithi007.stdout:3472 write 0x8873b83 thru 0x88828e5 (0xed63 bytes) 2024-09-12T09:31:04.911 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.932 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:04.931+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:04.933 INFO:teuthology.orchestra.run.smithi007.stdout:3475 trunc from 0xd53fbc9 to 0x21ff7e5 2024-09-12T09:31:04.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:04.988 INFO:teuthology.orchestra.run.smithi007.stdout:3476 writesame 0x9762f51 thru 0x976f097 data_size 0x1(0xc147 bytes) 2024-09-12T09:31:04.990 INFO:teuthology.orchestra.run.smithi007.stdout:3477 read 0x3b7eb0e thru 0x3b84685 (0x5b78 bytes) 2024-09-12T09:31:04.990 INFO:teuthology.orchestra.run.smithi007.stdout:3478 write 0x54cad89 thru 0x54d31ca (0x8442 bytes) 2024-09-12T09:31:04.998 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:04.998 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.000 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:04.999+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:05.014 INFO:teuthology.orchestra.run.smithi007.stdout:3482 write 0xebb5ee3 thru 0xebc1c96 (0xbdb4 bytes) 2024-09-12T09:31:05.021 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.021 INFO:teuthology.orchestra.run.smithi007.stdout:3484 read 0x70c5f36 thru 0x70c670a (0x7d5 bytes) 2024-09-12T09:31:05.021 INFO:teuthology.orchestra.run.smithi007.stdout:3485 read 0x9a62428 thru 0x9a71ba0 (0xf779 bytes) 2024-09-12T09:31:05.021 INFO:teuthology.orchestra.run.smithi007.stdout:3486 read 0x23243fe thru 0x233079d (0xc3a0 bytes) 2024-09-12T09:31:05.022 INFO:teuthology.orchestra.run.smithi007.stdout:3487 write 0x42f782 thru 0x434e26 (0x56a5 bytes) 2024-09-12T09:31:05.025 INFO:teuthology.orchestra.run.smithi007.stdout:3488 punch from 0x23b24ce to 0x23ba901, (0x8433 bytes) 2024-09-12T09:31:05.034 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:05.035 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:05.043 INFO:teuthology.orchestra.run.smithi007.stdout:3492 trunc from 0xebc1c97 to 0x29eeef6 2024-09-12T09:31:05.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:05.101 INFO:teuthology.orchestra.run.smithi007.stdout:3494 write 0xe0a5c12 thru 0xe0b2681 (0xca70 bytes) 2024-09-12T09:31:05.102 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:05.100+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:05.107 INFO:teuthology.orchestra.run.smithi007.stdout:3496 read 0x727d00e thru 0x7283e65 (0x6e58 bytes) 2024-09-12T09:31:05.108 INFO:teuthology.orchestra.run.smithi007.stdout:3497 punch from 0x740bb8 to 0x74be0a, (0xb252 bytes) 2024-09-12T09:31:05.108 INFO:teuthology.orchestra.run.smithi007.stdout:3498 writesame 0x6f9693b thru 0x6f9e728 data_size 0x9(0x7dee bytes) 2024-09-12T09:31:05.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:05.121 INFO:teuthology.orchestra.run.smithi007.stdout:3500 trunc from 0xe0b2682 to 0x925de0a 2024-09-12T09:31:05.130 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:05.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:31:05.151 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:05.154 INFO:teuthology.orchestra.run.smithi007.stdout:3501 trunc from 0x925de0a to 0x7c2705b 2024-09-12T09:31:05.169 INFO:teuthology.orchestra.run.smithi007.stdout:3502 read 0x59a26a3 thru 0x59aefc5 (0xc923 bytes) 2024-09-12T09:31:05.176 INFO:teuthology.orchestra.run.smithi007.stdout:3503 trunc from 0x7c2705b to 0xc43a533 2024-09-12T09:31:05.183 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:05.186 INFO:teuthology.orchestra.run.smithi007.stdout:3505 write 0xb3bfc4c thru 0xb3cc340 (0xc6f5 bytes) 2024-09-12T09:31:05.189 INFO:teuthology.orchestra.run.smithi007.stdout:3507 read 0x296d946 thru 0x29749c3 (0x707e bytes) 2024-09-12T09:31:05.189 INFO:teuthology.orchestra.run.smithi007.stdout:3511 read 0x30011bf thru 0x300725f (0x60a1 bytes) 2024-09-12T09:31:05.202 INFO:teuthology.orchestra.run.smithi007.stdout:3513 write 0xdaf073e thru 0xdaf3b2c (0x33ef bytes) 2024-09-12T09:31:05.203 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:05.199+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:05.209 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.209 INFO:teuthology.orchestra.run.smithi007.stdout:3517 read 0x11a2454 thru 0x11adba9 (0xb756 bytes) 2024-09-12T09:31:05.221 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:05.221 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:05.221 INFO:teuthology.orchestra.run.smithi007.stdout:3518 writesame 0xe7b4090 thru 0xe7bbda2 data_size 0xd(0x7d13 bytes) 2024-09-12T09:31:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:3519 punch from 0x41ecde to 0x420262, (0x1584 bytes) 2024-09-12T09:31:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:3520 punch from 0x7ca8980 to 0x7cb24d4, (0x9b54 bytes) 2024-09-12T09:31:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.222 INFO:teuthology.orchestra.run.smithi007.stdout:3523 writesame 0x10b5a50 thru 0x10bc673 data_size 0x12(0x6c24 bytes) 2024-09-12T09:31:05.225 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.225 INFO:teuthology.orchestra.run.smithi007.stdout:3526 writesame 0x13d31ca thru 0x13da8f6 data_size 0x1(0x772d bytes) 2024-09-12T09:31:05.242 INFO:teuthology.orchestra.run.smithi007.stdout:3527 writesame 0x6ee5696 thru 0x6ee9ef6 data_size 0x1(0x4861 bytes) 2024-09-12T09:31:05.260 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:05.261 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:05.261 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:05.261 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:05.261 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:05.262 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:05.266 INFO:teuthology.orchestra.run.smithi007.stdout:3528 punch from 0xd4d470e to 0xd4e0a7b, (0xc36d bytes) 2024-09-12T09:31:05.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:05.280 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:05.284 INFO:teuthology.orchestra.run.smithi007.stdout:3529 trunc from 0xe7bbda3 to 0x24b3362 2024-09-12T09:31:05.303 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:05.299+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:05.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:05.312 INFO:teuthology.orchestra.run.smithi007.stdout:3530 writesame 0x147edee thru 0x148c5fb data_size 0x5(0xd80e bytes) 2024-09-12T09:31:05.329 INFO:teuthology.orchestra.run.smithi007.stdout:3532 punch from 0x1853945 to 0x1855a7c, (0x2137 bytes) 2024-09-12T09:31:05.341 INFO:teuthology.orchestra.run.smithi007.stdout:3533 write 0x37ef062 thru 0x37f62b7 (0x7256 bytes) 2024-09-12T09:31:05.361 INFO:teuthology.orchestra.run.smithi007.stdout:3535 write 0x6783b06 thru 0x6792cca (0xf1c5 bytes) 2024-09-12T09:31:05.364 INFO:teuthology.orchestra.run.smithi007.stdout:3537 read 0x15d911f thru 0x15e270a (0x95ec bytes) 2024-09-12T09:31:05.365 INFO:teuthology.orchestra.run.smithi007.stdout:3539 writesame 0x1239842 thru 0x12450a5 data_size 0x2(0xb864 bytes) 2024-09-12T09:31:05.381 INFO:teuthology.orchestra.run.smithi007.stdout:3540 read 0xbd7b75 thru 0xbd859e (0xa2a bytes) 2024-09-12T09:31:05.381 INFO:teuthology.orchestra.run.smithi007.stdout:3541 read 0x20270c thru 0x20b889 (0x917e bytes) 2024-09-12T09:31:05.383 INFO:teuthology.orchestra.run.smithi007.stdout:3542 punch from 0xde48c7 to 0xdef481, (0xabba bytes) 2024-09-12T09:31:05.383 INFO:teuthology.orchestra.run.smithi007.stdout:3543 read 0x1f1e3c2 thru 0x1f24c6a (0x68a9 bytes) 2024-09-12T09:31:05.383 INFO:teuthology.orchestra.run.smithi007.stdout:3544 punch from 0x937c7d to 0x93d445, (0x57c8 bytes) 2024-09-12T09:31:05.383 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.383 INFO:teuthology.orchestra.run.smithi007.stdout:3546 read 0x5193c2c thru 0x519c642 (0x8a17 bytes) 2024-09-12T09:31:05.394 INFO:teuthology.orchestra.run.smithi007.stdout:3547 trunc from 0x6792ccb to 0x510d79 2024-09-12T09:31:05.404 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:05.404+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:05.427 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.287967717000001, 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:31:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:31:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186632175, 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:31:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186606722999997, 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186581871000001, 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:31:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:31:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186556533000001, 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:31:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186525502999999, 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:31:05.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186500342000002, 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:31:05.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186474442000005, 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:31:05.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186450836000006, 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:31:05.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:31:05.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:05.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186426323999996, 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:31:05.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186401885999999, 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:31:05.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186377299, 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:31:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:31:05.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186353283000003, 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:31:05.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186325440999994, 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:31:05.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:31:05.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186300595999995, 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:05.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186275870000003, 2024-09-12T09:31:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:31:05.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.186251373000005, 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:05.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:31:05.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.180445501999998, 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:31:05.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:31:05.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.035493537999997, 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:31:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:31:05.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:31:05.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.217996739999997, 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:05.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:05.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:31:05.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:05.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:05.531 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:05.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:05.533 INFO:teuthology.orchestra.run.smithi007.stdout:3548 write 0x8d2570e thru 0x8d33add (0xe3d0 bytes) 2024-09-12T09:31:05.534 INFO:teuthology.orchestra.run.smithi007.stdout:3550 write 0xcea4406 thru 0xceafc19 (0xb814 bytes) 2024-09-12T09:31:05.534 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.534 INFO:teuthology.orchestra.run.smithi007.stdout:3552 trunc from 0xceafc1a to 0x1c0debe 2024-09-12T09:31:05.534 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:05.531+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:05.587 INFO:teuthology.orchestra.run.smithi007.stdout:3553 write 0x77f3c25 thru 0x78033cd (0xf7a9 bytes) 2024-09-12T09:31:05.597 INFO:teuthology.orchestra.run.smithi007.stdout:3554 read 0x74b84e8 thru 0x74ba7c1 (0x22da bytes) 2024-09-12T09:31:05.598 INFO:teuthology.orchestra.run.smithi007.stdout:3555 writesame 0x5f9840 thru 0x5fd322 data_size 0x43(0x3ae3 bytes) 2024-09-12T09:31:05.603 INFO:teuthology.orchestra.run.smithi007.stdout:3557 punch from 0x15721 to 0x1a207, (0x4ae6 bytes) 2024-09-12T09:31:05.603 INFO:teuthology.orchestra.run.smithi007.stdout:3558 write 0x1db0e0d thru 0x1dba0bd (0x92b1 bytes) 2024-09-12T09:31:05.611 INFO:teuthology.orchestra.run.smithi007.stdout:3560 read 0x612a28a thru 0x612e57f (0x42f6 bytes) 2024-09-12T09:31:05.626 INFO:teuthology.orchestra.run.smithi007.stdout:3561 writesame 0x8a7ff71 thru 0x8a8b755 data_size 0x1(0xb7e5 bytes) 2024-09-12T09:31:05.626 INFO:teuthology.orchestra.run.smithi007.stdout:3562 write 0x48ca894 thru 0x48d7c2f (0xd39c bytes) 2024-09-12T09:31:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:3566 read 0x142df70 thru 0x143933e (0xb3cf bytes) 2024-09-12T09:31:05.631 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.634 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:05.632+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:05.650 INFO:teuthology.orchestra.run.smithi007.stdout:3569 write 0xc53d794 thru 0xc54c663 (0xeed0 bytes) 2024-09-12T09:31:05.658 INFO:teuthology.orchestra.run.smithi007.stdout:3570 read 0x3b6d50 thru 0x3c68cb (0xfb7c bytes) 2024-09-12T09:31:05.660 INFO:teuthology.orchestra.run.smithi007.stdout:3571 read 0x954185 thru 0x95f35e (0xb1da bytes) 2024-09-12T09:31:05.660 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.660 INFO:teuthology.orchestra.run.smithi007.stdout:3574 read 0x89d8766 thru 0x89e53e1 (0xcc7c bytes) 2024-09-12T09:31:05.660 INFO:teuthology.orchestra.run.smithi007.stdout:3575 writesame 0xa59f7b3 thru 0xa5af31d data_size 0x1(0xfb6b bytes) 2024-09-12T09:31:05.719 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:05.720 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:05.720 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:05.720 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:05.734 INFO:teuthology.orchestra.run.smithi007.stdout:3577 trunc from 0xc54c664 to 0x991d60e 2024-09-12T09:31:05.734 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:05.731+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:05.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:05.760 INFO:teuthology.orchestra.run.smithi007.stdout:3579 read 0xe72341 thru 0xe7a324 (0x7fe4 bytes) 2024-09-12T09:31:05.760 INFO:teuthology.orchestra.run.smithi007.stdout:3580 write 0x573ebbe thru 0x5744a4a (0x5e8d bytes) 2024-09-12T09:31:05.767 INFO:teuthology.orchestra.run.smithi007.stdout:3582 punch from 0x14bd70a to 0x14c3eb8, (0x67ae bytes) 2024-09-12T09:31:05.767 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.767 INFO:teuthology.orchestra.run.smithi007.stdout:3584 writesame 0x598f00f thru 0x599ae57 data_size 0x1(0xbe49 bytes) 2024-09-12T09:31:05.830 INFO:teuthology.orchestra.run.smithi007.stdout:3586 punch from 0x3f17dfa to 0x3f19db3, (0x1fb9 bytes) 2024-09-12T09:31:05.830 INFO:teuthology.orchestra.run.smithi007.stdout:3587 read 0x94b918e thru 0x94c1439 (0x82ac bytes) 2024-09-12T09:31:05.831 INFO:teuthology.orchestra.run.smithi007.stdout:3588 read 0x3f14b5d thru 0x3f1d48e (0x8932 bytes) 2024-09-12T09:31:05.832 INFO:teuthology.orchestra.run.smithi007.stdout:3589 writesame 0x980a7f2 thru 0x981525e data_size 0x1(0xaa6d bytes) 2024-09-12T09:31:05.835 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:05.832+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:05.867 INFO:teuthology.orchestra.run.smithi007.stdout:3590 write 0x64de148 thru 0x64dfb68 (0x1a21 bytes) 2024-09-12T09:31:05.872 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.872 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:05.888 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:05.888 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:05.888 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:05.889 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:05.889 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:3594 write 0xe2a3197 thru 0xe2a3f79 (0xde3 bytes) 2024-09-12T09:31:05.899 INFO:teuthology.orchestra.run.smithi007.stdout:3595 read 0xe22b938 thru 0xe22bbab (0x274 bytes) 2024-09-12T09:31:05.900 INFO:teuthology.orchestra.run.smithi007.stdout:3596 read 0x580bb55 thru 0x5817c47 (0xc0f3 bytes) 2024-09-12T09:31:05.902 INFO:teuthology.orchestra.run.smithi007.stdout:3597 write 0x811902a thru 0x8122b7c (0x9b53 bytes) 2024-09-12T09:31:05.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:05.910 INFO:teuthology.orchestra.run.smithi007.stdout:3598 trunc from 0xe2a3f7a to 0xd7b5ce9 2024-09-12T09:31:05.932 INFO:teuthology.orchestra.run.smithi007.stdout:3603 read 0x694fe4 thru 0x696e0d (0x1e2a bytes) 2024-09-12T09:31:05.934 INFO:teuthology.orchestra.run.smithi007.stdout:3606 trunc from 0xd7b5ce9 to 0xc47f510 2024-09-12T09:31:05.935 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:05.932+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:05.946 INFO:teuthology.orchestra.run.smithi007.stdout:3608 write 0x93b6330 thru 0x93c1a65 (0xb736 bytes) 2024-09-12T09:31:05.950 INFO:teuthology.orchestra.run.smithi007.stdout:3609 punch from 0x6e29a29 to 0x6e35690, (0xbc67 bytes) 2024-09-12T09:31:05.950 INFO:teuthology.orchestra.run.smithi007.stdout:3611 punch from 0x1ed7085 to 0x1ee316b, (0xc0e6 bytes) 2024-09-12T09:31:05.968 INFO:teuthology.orchestra.run.smithi007.stdout:3612 trunc from 0xc47f510 to 0xe3b51d 2024-09-12T09:31:06.033 INFO:teuthology.orchestra.run.smithi007.stdout:3613 write 0x276f9aa thru 0x277a5dc (0xac33 bytes) 2024-09-12T09:31:06.036 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:06.036+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:06.040 INFO:teuthology.orchestra.run.smithi007.stdout:3614 read 0x2320ac9 thru 0x2329640 (0x8b78 bytes) 2024-09-12T09:31:06.046 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:06.046 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:06.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:06.046 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.846752494, 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:31:06.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:31:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.831781765000002, 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:31:06.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:31:06.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:31:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514934760999999, 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:31:06.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514888401999997, 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:31:06.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:31:06.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:31:06.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514781182, 2024-09-12T09:31:06.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:31:06.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514687690999999, 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:06.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:31:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:31:06.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514621177999999, 2024-09-12T09:31:06.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:06.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:31:06.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514560369000002, 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:31:06.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:31:06.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:31:06.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514501076999998, 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:31:06.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514440567999998, 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:31:06.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:31:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:31:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:31:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:31:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:31:06.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:31:06.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514362513000002, 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:31:06.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:31:06.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:06.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:31:06.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:31:06.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:31:06.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514277137999997, 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:31:06.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514208818999997, 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:31:06.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:31:06.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:31:06.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:06.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514116233999999, 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:31:06.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:31:06.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:06.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514060325000003, 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:31:06.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.514028002000003, 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:31:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.513994408999999, 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:31:06.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.513943466999997, 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:31:06.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.513845312000001, 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:31:06.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.513788218000002, 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:31:06.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.165 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:06.166 INFO:teuthology.orchestra.run.smithi007.stdout:3615 write 0x2815f51 thru 0x2822096 (0xc146 bytes) 2024-09-12T09:31:06.166 INFO:teuthology.orchestra.run.smithi007.stdout:3616 trunc from 0x2822097 to 0x2566c70 2024-09-12T09:31:06.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:06.168 INFO:teuthology.orchestra.run.smithi007.stdout:3617 write 0x8cdbcf8 thru 0x8ce4403 (0x870c bytes) 2024-09-12T09:31:06.168 INFO:teuthology.orchestra.run.smithi007.stdout:3618 write 0xb170c63 thru 0xb17a7c2 (0x9b60 bytes) 2024-09-12T09:31:06.168 INFO:teuthology.orchestra.run.smithi007.stdout:3619 punch from 0x28b125 to 0x28f6bd, (0x4598 bytes) 2024-09-12T09:31:06.168 INFO:teuthology.orchestra.run.smithi007.stdout:3620 flatten 2024-09-12T09:31:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:31:06.169 INFO:teuthology.orchestra.run.smithi007.stdout:3621 writesame 0xa3f060c thru 0xa3f3ccb data_size 0x49(0x36c0 bytes) 2024-09-12T09:31:06.169 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:06.163+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:06.183 INFO:teuthology.orchestra.run.smithi007.stdout:3623 punch from 0x9e26f0b to 0x9e2bb1e, (0x4c13 bytes) 2024-09-12T09:31:06.200 INFO:teuthology.orchestra.run.smithi007.stdout:3625 trunc from 0xb17a7c3 to 0x3462052 2024-09-12T09:31:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:06.239 INFO:teuthology.orchestra.run.smithi007.stdout:3627 write 0x9dd9e94 thru 0x9ddced3 (0x3040 bytes) 2024-09-12T09:31:06.246 INFO:teuthology.orchestra.run.smithi007.stdout:3628 writesame 0x9564515 thru 0x956ed93 data_size 0x1(0xa87f bytes) 2024-09-12T09:31:06.267 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:06.264+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:06.298 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:06.298 INFO:teuthology.orchestra.run.smithi007.stdout:3630 punch from 0x3794f48 to 0x379880b, (0x38c3 bytes) 2024-09-12T09:31:06.298 INFO:teuthology.orchestra.run.smithi007.stdout:3632 writesame 0x3edf927 thru 0x3ee58c1 data_size 0xb(0x5f9b bytes) 2024-09-12T09:31:06.303 INFO:teuthology.orchestra.run.smithi007.stdout:3633 read 0x81350c2 thru 0x8144034 (0xef73 bytes) 2024-09-12T09:31:06.304 INFO:teuthology.orchestra.run.smithi007.stdout:3634 write 0x3ef1ae7 thru 0x3ef9361 (0x787b bytes) 2024-09-12T09:31:06.308 INFO:teuthology.orchestra.run.smithi007.stdout:3635 read 0x4e5f673 thru 0x4e6eafd (0xf48b bytes) 2024-09-12T09:31:06.308 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:06.314 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:06.314 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:06.322 INFO:teuthology.orchestra.run.smithi007.stdout:3637 write 0xbc1e547 thru 0xbc25de4 (0x789e bytes) 2024-09-12T09:31:06.325 INFO:teuthology.orchestra.run.smithi007.stdout:3639 punch from 0x9af869f to 0x9b033be, (0xad1f bytes) 2024-09-12T09:31:06.325 INFO:teuthology.orchestra.run.smithi007.stdout:3640 writesame 0x46d5730 thru 0x46de688 data_size 0x1(0x8f59 bytes) 2024-09-12T09:31:06.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:06.372 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:06.372+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:06.375 INFO:teuthology.orchestra.run.smithi007.stdout:3641 read 0x3db4795 thru 0x3dbb88a (0x70f6 bytes) 2024-09-12T09:31:06.391 INFO:teuthology.orchestra.run.smithi007.stdout:3642 trunc from 0xbc25de5 to 0x56d4da4 2024-09-12T09:31:06.412 INFO:teuthology.orchestra.run.smithi007.stdout:3643 writesame 0xa20edc thru 0xa2f18e data_size 0x9f(0xe2b3 bytes) 2024-09-12T09:31:06.435 INFO:teuthology.orchestra.run.smithi007.stdout:3644 write 0xb0c38b2 thru 0xb0c7339 (0x3a88 bytes) 2024-09-12T09:31:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:3647 write 0xc21aee7 thru 0xc2261c2 (0xb2dc bytes) 2024-09-12T09:31:06.459 INFO:teuthology.orchestra.run.smithi007.stdout:3648 writesame 0x49e3af9 thru 0x49f0368 data_size 0x18(0xc870 bytes) 2024-09-12T09:31:06.469 INFO:teuthology.orchestra.run.smithi007.stdout:3650 read 0x8d9d721 thru 0x8da5ef1 (0x87d1 bytes) 2024-09-12T09:31:06.469 INFO:teuthology.orchestra.run.smithi007.stdout:3651 punch from 0x736fa5b to 0x737257b, (0x2b20 bytes) 2024-09-12T09:31:06.469 INFO:teuthology.orchestra.run.smithi007.stdout:3652 write 0x130715c thru 0x130a151 (0x2ff6 bytes) 2024-09-12T09:31:06.470 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:06.467+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:06.472 INFO:teuthology.orchestra.run.smithi007.stdout:3653 read 0x8ec4395 thru 0x8ed08bd (0xc529 bytes) 2024-09-12T09:31:06.473 INFO:teuthology.orchestra.run.smithi007.stdout:3654 clone 35 order 18 su 16384 sc 4 2024-09-12T09:31:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:06.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:06.570 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:06.567+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:06.671 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:06.668+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:06.689 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:06.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:06.689 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.388255610000002, 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:06.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.205836691999998, 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:06.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.813050179000001, 2024-09-12T09:31:06.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:06.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:06.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.465520290999997, 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:06.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.463066380999997, 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:06.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:06.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.714 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.460956494000001, 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:06.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.965209849000001, 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:06.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.934801563000001, 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:06.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.713837004, 2024-09-12T09:31:06.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:06.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:06.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:06.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0147257209999996, 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:06.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:06.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.0147087920000004, 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:06.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9759471939999997, 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:06.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:06.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:06.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9185787049999998, 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:06.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:06.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9185552860000001, 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:06.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:06.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:06.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.887326281, 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8062233240000003, 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:06.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:06.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:06.784 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8061973289999997, 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:06.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5529808410000001, 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:06.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:06.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3302967649999999, 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3302699569999996, 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:06.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:06.812 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:06.808+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:06.912 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:06.912+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:06.990 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:06.990 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:07.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:07.013 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:07.011+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:07.114 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:07.112+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:07.152 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 0, 7] out_osds: [6, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:07.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:07.153 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [2, 3, 4, 5, 0, 7] 2024-09-12T09:31:07.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-12T09:31:07.182 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:07.182 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:07.182 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:07.182 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:07.182 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:07.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:07.215 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:07.211+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:07.316 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:07.315+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:07.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:07.368 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148930 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 937984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9313844719999995, 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097427275999999993, 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148930 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435664+0000", 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435666+0000", 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435672+0000", 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4969999999999998e-06 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435677+0000", 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1780000000000002e-06 2024-09-12T09:31:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532783+0000", 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097105891 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532861+0000", 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7760000000000001e-05 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533090+0000", 2024-09-12T09:31:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022925500000000001 2024-09-12T09:31:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9313632959999998, 2024-09-12T09:31:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9313380650000003, 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9313001449999998, 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.931270144, 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:07.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9232357230000003, 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9229148499999997, 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9224910240000002, 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9224395449999996, 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9224107969999995, 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9218603309999995, 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9218175090000003, 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9217716180000002, 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9212600599999998, 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9212122809999999, 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.921156635, 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.921124614, 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9205604650000003, 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.9201665639999996, 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:07.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.919774866, 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:07.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:07.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:07.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:07.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:07.490 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:07.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:07.492 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-12T09:31:07.492 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:07.488+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:07.529 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:07.531 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:31:07.528+0000 7f3454e55640 -1 mon.a@0(leader).osd e100 definitely_dead 0 2024-09-12T09:31:07.563 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:07.563 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:07.591 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:07.587+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:07.593 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:07.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:07.621 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:07.645 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:07.646 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:07.671 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:07.687 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:07.688 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:07.692 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:07.691+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:07.692 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:07.706 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:07.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:07.710 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:07.790 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:07.790 INFO:teuthology.orchestra.run.smithi007.stdout:3656 flatten 2024-09-12T09:31:07.792 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:07.791+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:07.865 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:07.865 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:07.879 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:07.894 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:07.892+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:07.995 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:07.991+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:08.050 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.050 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:08.112 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:08.111+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.0. 2024-09-12T09:31:08.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:31:08.197 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:08.196+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:08.239 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.239 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:08.300 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:08.299+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:08.375 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.376 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:08.398 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:08.396+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:08.499 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:08.496+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:08.549 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.550 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:08.599 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:08.596+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:08.687 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.687 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.700 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:08.700+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:08.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:08.801 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:08.799+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:08.817 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:08.817 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:08.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:08.901 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:08.900+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.002 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.002 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:08.999+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:09.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:09.103 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:09.100+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:31:09.186 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.186 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:09.244 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:09.244+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:09.305 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:09.303+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:09.318 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.318 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:09.405 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:09.404+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:09.501 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.501 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.506 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:09.503+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:09.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:09.607 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:09.604+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:09.650 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.650 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:09.708 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:09.703+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:09.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:09.809 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:09.807+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:09.839 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:09.839 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:09.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:09.899 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:09.910 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:09.907+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:09.916 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:09.933 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:09.950 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:09.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:09.972 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:09.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:09.975 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:09.975 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.837486166000005, 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:09.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:09.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:31:09.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:09.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:31:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:09.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736150624000004, 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:31:09.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:31:09.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736125172000001, 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:09.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:09.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:31:09.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736100320000006, 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:09.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:09.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:31:09.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:09.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736074982000005, 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:09.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:31:10.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736043952000003, 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:10.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:31:10.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.736018791000006, 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:31:10.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:31:10.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735992890999995, 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:31:10.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:10.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735969284999996, 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:31:10.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:31:10.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:31:10.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735944773, 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:31:10.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:31:10.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735920335000003, 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.032 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:31:10.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735895748000004, 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:31:10.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735871732000007, 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:31:10.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735843889999998, 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:31:10.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735819045, 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:31:10.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735794319000007, 2024-09-12T09:31:10.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:10.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:31:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.735769821999995, 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:31:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.729963951000002, 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:31:10.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:10.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.585011987000001, 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:31:10.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:10.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.767515189000001, 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:31:10.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:31:10.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.083 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:10.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:10.084 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:10.084+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:10.185 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:10.184+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:10.235 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:10.265 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:10.285 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:10.283+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:10.297 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:10.305 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:10.306 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:10.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:10.387 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:10.383+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:10.487 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:10.483+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:10.558 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:10.558 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:10.559 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:10.559 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:10.559 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:10.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:10.588 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:10.587+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:10.688 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:10.687+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.729 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.528286508000001, 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.730 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:31:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:31:10.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:31:10.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:31:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.513315779000003, 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:31:10.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196468775, 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:31:10.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:31:10.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196422415999997, 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:10.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:31:10.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196315196, 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:31:10.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196221704999999, 2024-09-12T09:31:10.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:10.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:31:10.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:31:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196155191999999, 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:31:10.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196094383000002, 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:31:10.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.196035090999999, 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:31:10.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195974581999998, 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:31:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:31:10.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195896527000002, 2024-09-12T09:31:10.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:10.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195811151999997, 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:31:10.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:31:10.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:31:10.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195742832999997, 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195650248, 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:31:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195594339000003, 2024-09-12T09:31:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:31:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195562015999997, 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:31:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195528422999999, 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:31:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:31:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195477480999998, 2024-09-12T09:31:10.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:31:10.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195379326000001, 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.195322232000002, 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:10.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:31:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:10.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:10.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:10.832 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:10.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:10.834 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:10.832+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:10.934 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:10.932+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:11.030 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:11.030 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:11.031 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:11.035 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:11.032+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:11.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:11.135 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:11.131+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:11.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:31:11.237 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:11.235+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:11.262 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:11.262 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:11.263 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:11.263 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:11.263 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:11.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:11.338 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:11.336+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:11.403 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.101555200999996, 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:11.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:11.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919136283, 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:11.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.526349770000003, 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:11.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:11.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.178819881999999, 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.176365971999999, 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.174256085000003, 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.678509439999999, 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.648101153999999, 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.427136595, 2024-09-12T09:31:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.728025312, 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.728008383000001, 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.689246785, 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.631878296, 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.631854877, 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.600625872, 2024-09-12T09:31:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.519522915, 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.51949692, 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.266280432, 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.043596356, 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.043569548000001, 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:11.515 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:11.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:11.516 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:11.516+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:11.617 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:11.616+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:11.689 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:11.689 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:11.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:11.744 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:11.744+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:11.819 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:11.819+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:11.873 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:11.873 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:11.873 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:11.873 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:11.873 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:11.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:11.920 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:11.919+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148930 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 937984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.581214062000001, 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097427275999999993, 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148930 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435664+0000", 2024-09-12T09:31:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435666+0000", 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435672+0000", 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4969999999999998e-06 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435677+0000", 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1780000000000002e-06 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532783+0000", 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097105891 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532861+0000", 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7760000000000001e-05 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533090+0000", 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022925500000000001 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.581192886, 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.581167655, 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.581129734999999, 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.581099734, 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.573065313000001, 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.572744439999999, 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.572320614000001, 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.572269134999999, 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.572240387000001, 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.571689921000001, 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.571647099, 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.571601208000001, 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.571089649999999, 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.571041871, 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.570986225, 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.570954204, 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.570390055000001, 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.569996154, 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.569604456, 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:12.115 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:12.117 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:12.117 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:12.115+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:12.206 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:31:12.219 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:12.216+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:12.276 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:12.276 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:12.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171658 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260778+0000", 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0074284809999999998, 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074534390000000001, 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171658 2024-09-12T09:31:12.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260778+0000", 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260778+0000", 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260779+0000", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5200000000000002e-07 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260781+0000", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2469999999999998e-06 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260788+0000", 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6599999999999998e-06 2024-09-12T09:31:12.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260793+0000", 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4970000000000001e-06 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171690 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263977+0000", 2024-09-12T09:31:12.280 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042295459999999998, 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043247490000000001, 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171690 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263977+0000", 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263977+0000", 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263978+0000", 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263982+0000", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7960000000000002e-06 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263993+0000", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1858999999999999e-05 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264002+0000", 2024-09-12T09:31:12.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1699999999999997e-06 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171722 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266691+0000", 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015152709999999999, 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001671348, 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.284 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171722 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266691+0000", 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266691+0000", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266692+0000", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-07 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266695+0000", 2024-09-12T09:31:12.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6819999999999999e-06 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266703+0000", 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2369999999999992e-06 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266711+0000", 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171659 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260802+0000", 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.007404524, 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076200139999999996, 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171659 2024-09-12T09:31:12.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260802+0000", 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260802+0000", 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260803+0000", 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-07 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260805+0000", 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:31:12.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260811+0000", 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260816+0000", 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.843e-06 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171691 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264014+0000", 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0041926539999999997, 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044660730000000001, 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171691 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264014+0000", 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264014+0000", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264015+0000", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7899999999999998e-07 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264017+0000", 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6769999999999999e-06 2024-09-12T09:31:12.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264025+0000", 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2490000000000004e-06 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264030+0000", 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.925e-06 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171723 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266720+0000", 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0014869569999999999, 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001820739, 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171723 2024-09-12T09:31:12.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266720+0000", 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266720+0000", 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266720+0000", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4099999999999998e-07 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266723+0000", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4849999999999999e-06 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266731+0000", 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1089999999999998e-06 2024-09-12T09:31:12.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266737+0000", 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5429999999999999e-06 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171660 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260824+0000", 2024-09-12T09:31:12.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073830639999999999, 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077735920000000002, 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171660 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260824+0000", 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260824+0000", 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260824+0000", 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260826+0000", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.092e-06 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260833+0000", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260838+0000", 2024-09-12T09:31:12.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171692 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264039+0000", 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0041672799999999998, 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046159870000000002, 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171692 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264039+0000", 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264039+0000", 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264040+0000", 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4600000000000005e-07 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264042+0000", 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3070000000000001e-06 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264049+0000", 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0119999999999999e-06 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264054+0000", 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1279999999999999e-06 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171724 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266748+0000", 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0014584159999999999, 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019624410000000001, 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171724 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266748+0000", 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266748+0000", 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266749+0000", 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6400000000000002e-07 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266751+0000", 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.458e-06 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266759+0000", 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266779+0000", 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9633e-05 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171661 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260844+0000", 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073624670000000001, 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079358439999999992, 2024-09-12T09:31:12.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171661 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260844+0000", 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260844+0000", 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260845+0000", 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260846+0000", 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.739e-06 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260853+0000", 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3400000000000003e-06 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260857+0000", 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8280000000000002e-06 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171693 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264063+0000", 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0041438849999999999, 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004776392, 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171693 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264063+0000", 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264063+0000", 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264063+0000", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2900000000000004e-07 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264066+0000", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3250000000000002e-06 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264077+0000", 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1073000000000001e-05 2024-09-12T09:31:12.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264083+0000", 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9819999999999996e-06 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171725 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266789+0000", 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001417895, 2024-09-12T09:31:12.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002107364, 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171725 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266789+0000", 2024-09-12T09:31:12.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266789+0000", 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266789+0000", 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3499999999999996e-07 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266792+0000", 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9100000000000001e-06 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266801+0000", 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2579999999999995e-06 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266808+0000", 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4039999999999998e-06 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171662 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260864+0000", 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073427329999999997, 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080888139999999997, 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171662 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260864+0000", 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260864+0000", 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260864+0000", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260870+0000", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4999999999999999e-06 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260876+0000", 2024-09-12T09:31:12.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5339999999999996e-06 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260881+0000", 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1000000000000003e-06 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171694 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264097+0000", 2024-09-12T09:31:12.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0041097149999999999, 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049119100000000002, 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171694 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264097+0000", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264097+0000", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264098+0000", 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8800000000000002e-07 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264100+0000", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6460000000000002e-06 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264108+0000", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5229999999999998e-06 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264113+0000", 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6559999999999997e-06 2024-09-12T09:31:12.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171726 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266838+0000", 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00136882, 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002226708, 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171726 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266838+0000", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266838+0000", 2024-09-12T09:31:12.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266839+0000", 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9599999999999998e-07 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266843+0000", 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2220000000000004e-06 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266852+0000", 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6190000000000006e-06 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266860+0000", 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6410000000000002e-06 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171663 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260890+0000", 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073170910000000004, 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082314290000000002, 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171663 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260890+0000", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260890+0000", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260890+0000", 2024-09-12T09:31:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.08e-07 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260892+0000", 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9379999999999999e-06 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260900+0000", 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9189999999999999e-06 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260905+0000", 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7709999999999997e-06 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171695 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264121+0000", 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.004085399, 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050565369999999998, 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171695 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264121+0000", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264121+0000", 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264122+0000", 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999996e-07 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264124+0000", 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2639999999999998e-06 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264131+0000", 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.691e-06 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264136+0000", 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1429999999999997e-06 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171727 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266872+0000", 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0013345080000000001, 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023669530000000002, 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171727 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266872+0000", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266872+0000", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266873+0000", 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7800000000000001e-07 2024-09-12T09:31:12.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266876+0000", 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0319999999999999e-06 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266885+0000", 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7220000000000005e-06 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266891+0000", 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3199999999999996e-06 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171664 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260912+0000", 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0072944739999999996, 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083847890000000001, 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171664 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260912+0000", 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260912+0000", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260913+0000", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0100000000000005e-07 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260915+0000", 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0650000000000001e-06 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260921+0000", 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3090000000000001e-06 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260926+0000", 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171696 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264143+0000", 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0040634470000000004, 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005210877, 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171696 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264143+0000", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264143+0000", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264144+0000", 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6499999999999999e-07 2024-09-12T09:31:12.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264146+0000", 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1229999999999998e-06 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264154+0000", 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8089999999999993e-06 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264160+0000", 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.049e-06 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171728 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266902+0000", 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0013049820000000001, 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025090429999999999, 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171728 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266902+0000", 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266902+0000", 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266902+0000", 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266905+0000", 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7609999999999999e-06 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266914+0000", 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.602e-06 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266920+0000", 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5690000000000001e-06 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171665 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.260935+0000", 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0072719459999999996, 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085346539999999992, 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171665 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260935+0000", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260935+0000", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260935+0000", 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:31:12.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260938+0000", 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260946+0000", 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8979999999999996e-06 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.260950+0000", 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171697 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.264167+0000", 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0040398869999999998, 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053597339999999997, 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171697 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264167+0000", 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264167+0000", 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264167+0000", 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000002e-07 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264170+0000", 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3089999999999998e-06 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264176+0000", 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9489999999999999e-06 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.264182+0000", 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1279999999999999e-06 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171729 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266929+0000", 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001277825, 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026541540000000001, 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171729 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266929+0000", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266929+0000", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266929+0000", 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1799999999999995e-07 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266932+0000", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266940+0000", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0309999999999998e-06 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266947+0000", 2024-09-12T09:31:12.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2559999999999999e-06 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171666 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.262873+0000", 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00533388, 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067699470000000001, 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171666 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262873+0000", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262873+0000", 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262874+0000", 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.186e-06 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262880+0000", 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8499999999999999e-06 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262893+0000", 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3319000000000001e-05 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262902+0000", 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1379999999999996e-06 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171698 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265776+0000", 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0024305989999999999, 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039248440000000002, 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171698 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265776+0000", 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265776+0000", 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265777+0000", 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.023e-06 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265782+0000", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.011e-06 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265792+0000", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4229999999999994e-06 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265799+0000", 2024-09-12T09:31:12.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171730 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268796+0000", 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9994102, 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00096502100000000004, 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171730 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268796+0000", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268796+0000", 2024-09-12T09:31:12.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268797+0000", 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.096e-06 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268802+0000", 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7690000000000004e-06 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268813+0000", 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1112e-05 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268821+0000", 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3939999999999998e-06 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171667 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.262922+0000", 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0052843760000000004, 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068971329999999997, 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171667 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262922+0000", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262922+0000", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262923+0000", 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0100000000000004e-07 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262926+0000", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262935+0000", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6920000000000008e-06 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262941+0000", 2024-09-12T09:31:12.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.455e-06 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171699 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265813+0000", 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023940369999999999, 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040662240000000002, 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171699 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265813+0000", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265813+0000", 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265813+0000", 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1699999999999997e-07 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265816+0000", 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.734e-06 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265823+0000", 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265829+0000", 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.558e-06 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171731 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268834+0000", 2024-09-12T09:31:12.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9993725, 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001102823, 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171731 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268834+0000", 2024-09-12T09:31:12.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268834+0000", 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268835+0000", 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4499999999999997e-07 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268838+0000", 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.655e-06 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268845+0000", 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3329999999999999e-06 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268852+0000", 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7909999999999999e-06 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171668 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.262951+0000", 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0052558680000000003, 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070494240000000003, 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171668 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262951+0000", 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262951+0000", 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262951+0000", 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262954+0000", 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.492e-06 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262961+0000", 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3860000000000001e-06 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262967+0000", 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171700 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265838+0000", 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023690540000000002, 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042198089999999997, 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171700 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265838+0000", 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265838+0000", 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265838+0000", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2900000000000004e-07 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265841+0000", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4109999999999999e-06 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265848+0000", 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6310000000000003e-06 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265853+0000", 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171732 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268860+0000", 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9993467, 2024-09-12T09:31:12.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001255677, 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171732 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268860+0000", 2024-09-12T09:31:12.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268860+0000", 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268860+0000", 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268863+0000", 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2819999999999999e-06 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268869+0000", 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6039999999999998e-06 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268875+0000", 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171669 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.262976+0000", 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0052307609999999996, 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007196895, 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171669 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262976+0000", 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262976+0000", 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262977+0000", 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3600000000000003e-07 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262979+0000", 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3470000000000001e-06 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262987+0000", 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6669999999999996e-06 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.262993+0000", 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9200000000000001e-06 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171701 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265862+0000", 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023451380000000001, 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043775000000000003, 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171701 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265862+0000", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265862+0000", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265862+0000", 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:31:12.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265864+0000", 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0640000000000001e-06 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265870+0000", 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6210000000000001e-06 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265875+0000", 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.045e-06 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171733 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268893+0000", 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9993143, 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001403385, 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171733 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268893+0000", 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268893+0000", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268893+0000", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6199999999999997e-07 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268896+0000", 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4420000000000001e-06 2024-09-12T09:31:12.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268904+0000", 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3280000000000006e-06 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268909+0000", 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171670 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263023+0000", 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0051837890000000003, 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073300379999999997, 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171670 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.407 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263023+0000", 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263023+0000", 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263023+0000", 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0500000000000003e-07 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263031+0000", 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9510000000000006e-06 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263043+0000", 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1090999999999999e-05 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263049+0000", 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9319999999999999e-06 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171702 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265882+0000", 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002324577, 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045279059999999999, 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171702 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265882+0000", 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265882+0000", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265883+0000", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4799999999999998e-07 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265889+0000", 2024-09-12T09:31:12.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999997e-06 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265896+0000", 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9020000000000002e-06 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265902+0000", 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3210000000000004e-06 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171734 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268917+0000", 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9992895, 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001549805, 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171734 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268917+0000", 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268917+0000", 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268918+0000", 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4899999999999995e-07 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268920+0000", 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1579999999999999e-06 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268926+0000", 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1310000000000003e-06 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268931+0000", 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1919999999999996e-06 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171671 3.14 3:29ab3cfe:::rbd_data.1191e0d4a936.0000000000000003:25 [read 13991936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263059+0000", 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0051476940000000004, 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074891189999999998, 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171671 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263059+0000", 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263059+0000", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263060+0000", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8299999999999997e-07 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263062+0000", 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.413e-06 2024-09-12T09:31:12.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263070+0000", 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6699999999999994e-06 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263076+0000", 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0039999999999996e-06 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270523+0000", 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074470480000000004 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171703 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265945+0000", 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022614279999999998, 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046662630000000004, 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171703 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265945+0000", 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265945+0000", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265946+0000", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9999999999999995e-07 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265952+0000", 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3770000000000004e-06 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265961+0000", 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7799999999999995e-06 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265970+0000", 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1519999999999992e-06 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.426 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171735 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268941+0000", 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9992661, 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017309300000000001, 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171735 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268941+0000", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268941+0000", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268941+0000", 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-12T09:31:12.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268943+0000", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2529999999999999e-06 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268950+0000", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8369999999999999e-06 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268956+0000", 2024-09-12T09:31:12.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2399999999999998e-06 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171672 3.14 3:29ab3cfe:::rbd_data.1191e0d4a936.0000000000000003:25 [read 14057472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263084+0000", 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0051227879999999996, 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076550619999999998, 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171672 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263084+0000", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263084+0000", 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263085+0000", 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263087+0000", 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263094+0000", 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4159999999999998e-06 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263100+0000", 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8629999999999996e-06 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270684+0000", 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075835629999999998 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270719+0000", 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5148999999999998e-05 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171704 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.265989+0000", 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022176909999999999, 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048201579999999997, 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171704 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265989+0000", 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265989+0000", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265990+0000", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1900000000000002e-07 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.265993+0000", 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.078e-06 2024-09-12T09:31:12.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266002+0000", 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9600000000000006e-06 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266009+0000", 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0380000000000002e-06 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171736 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268963+0000", 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9992437, 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018971050000000001, 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171736 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.438 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268963+0000", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268963+0000", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268963+0000", 2024-09-12T09:31:12.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4700000000000002e-07 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268965+0000", 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.029e-06 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268971+0000", 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8660000000000002e-06 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268976+0000", 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0429999999999998e-06 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171673 3.14 3:29ab3cfe:::rbd_data.1191e0d4a936.0000000000000003:25 [read 14123008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263109+0000", 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0050975350000000003, 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007804465, 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171673 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263109+0000", 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263109+0000", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263110+0000", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3200000000000005e-07 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263112+0000", 2024-09-12T09:31:12.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.514e-06 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263120+0000", 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263126+0000", 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1279999999999996e-06 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270826+0000", 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077002779999999996 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171705 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266019+0000", 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021877569999999998, 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049521089999999997, 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171705 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266019+0000", 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266019+0000", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266019+0000", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266022+0000", 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6390000000000001e-06 2024-09-12T09:31:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266045+0000", 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2676000000000001e-05 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266053+0000", 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2570000000000007e-06 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171737 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.268996+0000", 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9992104, 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020186840000000002, 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171737 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.449 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268996+0000", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268996+0000", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268997+0000", 2024-09-12T09:31:12.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9500000000000002e-07 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.268999+0000", 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2350000000000002e-06 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269006+0000", 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9399999999999996e-06 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269013+0000", 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9380000000000003e-06 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171674 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263134+0000", 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0050730020000000001, 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079414389999999998, 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.452 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171674 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263134+0000", 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263134+0000", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263134+0000", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3600000000000004e-07 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263136+0000", 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2520000000000002e-06 2024-09-12T09:31:12.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263143+0000", 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8800000000000002e-06 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263149+0000", 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7509999999999997e-06 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171706 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266064+0000", 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021422429999999998, 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050683530000000003, 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171706 2024-09-12T09:31:12.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266064+0000", 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266064+0000", 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266065+0000", 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1600000000000001e-07 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266068+0000", 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3510000000000002e-06 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266081+0000", 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3016e-05 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266089+0000", 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1910000000000003e-06 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171738 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269021+0000", 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9991856, 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021713510000000002, 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.459 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171738 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269021+0000", 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269021+0000", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269021+0000", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9599999999999999e-07 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269023+0000", 2024-09-12T09:31:12.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9989999999999998e-06 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269031+0000", 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0940000000000002e-06 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269036+0000", 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171675 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263157+0000", 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0050493559999999996, 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080969230000000007, 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171675 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263157+0000", 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263157+0000", 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263158+0000", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263161+0000", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6919999999999998e-06 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263168+0000", 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5140000000000004e-06 2024-09-12T09:31:12.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263174+0000", 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0249999999999999e-06 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.271260+0000", 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080860619999999998 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171707 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266099+0000", 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021071810000000001, 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052191060000000003, 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171707 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.467 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266099+0000", 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266099+0000", 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266100+0000", 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8299999999999997e-07 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266103+0000", 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.926e-06 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266112+0000", 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0920000000000004e-06 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266119+0000", 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7669999999999998e-06 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171739 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269043+0000", 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9991636, 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023344490000000002, 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.470 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171739 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269043+0000", 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269043+0000", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269043+0000", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269046+0000", 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3470000000000001e-06 2024-09-12T09:31:12.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269053+0000", 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0400000000000004e-06 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269058+0000", 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9840000000000001e-06 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171676 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263182+0000", 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0050244770000000003, 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082567490000000007, 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171676 2024-09-12T09:31:12.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263182+0000", 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263182+0000", 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263183+0000", 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0399999999999995e-07 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263185+0000", 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263193+0000", 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4710000000000001e-06 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263199+0000", 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9290000000000003e-06 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171708 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266128+0000", 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002078894, 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053691199999999998, 2024-09-12T09:31:12.477 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171708 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266128+0000", 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266128+0000", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266128+0000", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4099999999999999e-07 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266131+0000", 2024-09-12T09:31:12.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.627e-06 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266139+0000", 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4440000000000008e-06 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266146+0000", 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3110000000000002e-06 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171740 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269087+0000", 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9991193, 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002468736, 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171740 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269087+0000", 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269087+0000", 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269088+0000", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3499999999999996e-07 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269091+0000", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5339999999999998e-06 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269098+0000", 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-09-12T09:31:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269104+0000", 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4110000000000001e-06 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171677 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263225+0000", 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0049816820000000003, 2024-09-12T09:31:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0083931690000000007, 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171677 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263225+0000", 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263225+0000", 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263226+0000", 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2900000000000002e-07 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263230+0000", 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1139999999999999e-06 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263240+0000", 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263247+0000", 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0690000000000004e-06 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171709 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266155+0000", 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002051172, 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055208239999999997, 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171709 2024-09-12T09:31:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266155+0000", 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266155+0000", 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266156+0000", 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266159+0000", 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266175+0000", 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5695000000000001e-05 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266181+0000", 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6470000000000001e-06 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171741 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269114+0000", 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9990921, 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026210790000000001, 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171741 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269114+0000", 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269114+0000", 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269115+0000", 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4600000000000004e-07 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269122+0000", 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2189999999999998e-06 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269129+0000", 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269134+0000", 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171678 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263260+0000", 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0049470779999999997, 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085372369999999996, 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171678 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263260+0000", 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263260+0000", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263260+0000", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2300000000000001e-07 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263263+0000", 2024-09-12T09:31:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4609999999999998e-06 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263270+0000", 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3089999999999998e-06 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263276+0000", 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3489999999999997e-06 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171710 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266188+0000", 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020189320000000002, 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056675720000000001, 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171710 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266188+0000", 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266188+0000", 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266188+0000", 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266190+0000", 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.668e-06 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266194+0000", 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6949999999999999e-06 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266198+0000", 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5930000000000002e-06 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171742 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269141+0000", 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9990654, 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027727379999999999, 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171742 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269141+0000", 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269141+0000", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269142+0000", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8599999999999998e-07 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269144+0000", 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:31:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269151+0000", 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269155+0000", 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9470000000000002e-06 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171679 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263284+0000", 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0049223770000000003, 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086905390000000006, 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171679 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263284+0000", 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263284+0000", 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263285+0000", 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000005e-07 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263287+0000", 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2759999999999999e-06 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263294+0000", 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0690000000000004e-06 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263300+0000", 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4049999999999999e-06 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.271962+0000", 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086621049999999998 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171711 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266203+0000", 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020039010000000002, 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058370799999999997, 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171711 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266203+0000", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266203+0000", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266203+0000", 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266205+0000", 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.483e-06 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266209+0000", 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3810000000000001e-06 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266213+0000", 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4399999999999998e-06 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171743 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269162+0000", 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9990444, 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002937678, 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171743 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269162+0000", 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269162+0000", 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269162+0000", 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269164+0000", 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9e-06 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269171+0000", 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6279999999999999e-06 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269176+0000", 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9030000000000003e-06 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171680 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263308+0000", 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048989100000000002, 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088522640000000003, 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171680 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263308+0000", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263308+0000", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263308+0000", 2024-09-12T09:31:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263310+0000", 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1280000000000002e-06 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263317+0000", 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3260000000000001e-06 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263322+0000", 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2510000000000002e-06 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171712 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266218+0000", 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0019886449999999998, 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060026899999999998, 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171712 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266218+0000", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266218+0000", 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266218+0000", 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266220+0000", 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5629999999999999e-06 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266224+0000", 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3510000000000004e-06 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266229+0000", 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171744 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269182+0000", 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9990244, 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031909299999999998, 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171744 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269182+0000", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269182+0000", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269183+0000", 2024-09-12T09:31:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2199999999999999e-07 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269185+0000", 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.024e-06 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269191+0000", 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-06 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269196+0000", 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7260000000000002e-06 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171681 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263329+0000", 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048773480000000001, 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090821930000000006, 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171681 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263329+0000", 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263329+0000", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263330+0000", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263332+0000", 2024-09-12T09:31:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5059999999999998e-06 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263339+0000", 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263344+0000", 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1930000000000001e-06 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.272412+0000", 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090679880000000008 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171713 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266234+0000", 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0019727410000000001, 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062459760000000003, 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171713 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266234+0000", 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266234+0000", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266234+0000", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3799999999999998e-07 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266236+0000", 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.469e-06 2024-09-12T09:31:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266240+0000", 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1200000000000004e-06 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266244+0000", 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171745 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269203+0000", 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9990039, 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033374920000000001, 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171745 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269203+0000", 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269203+0000", 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269203+0000", 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269205+0000", 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0899999999999999e-06 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269212+0000", 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3879999999999997e-06 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269217+0000", 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9189999999999998e-06 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171682 3.14 3:29ab3cfe:::rbd_data.1191e0d4a936.0000000000000003:25 [read 13680640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263351+0000", 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048552819999999998, 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092579240000000007, 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171682 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263351+0000", 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263351+0000", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263352+0000", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263360+0000", 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8310000000000001e-06 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263367+0000", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4070000000000004e-06 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263373+0000", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4809999999999997e-06 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.272583+0000", 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092107850000000008 2024-09-12T09:31:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171714 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266437+0000", 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017693609999999999, 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062335050000000003, 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171714 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266437+0000", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266437+0000", 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266438+0000", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8800000000000001e-07 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266442+0000", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.822e-06 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266449+0000", 2024-09-12T09:31:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3180000000000001e-06 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266454+0000", 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1270000000000002e-06 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171746 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.269977+0000", 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.998229, 2024-09-12T09:31:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002753545, 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171746 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269977+0000", 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269977+0000", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269978+0000", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1800000000000004e-07 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269983+0000", 2024-09-12T09:31:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1540000000000004e-06 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.269994+0000", 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1019000000000001e-05 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270001+0000", 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171683 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263380+0000", 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048263109999999998, 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00941759, 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171683 2024-09-12T09:31:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263380+0000", 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263380+0000", 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263381+0000", 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263383+0000", 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0389999999999999e-06 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263389+0000", 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3319999999999997e-06 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263394+0000", 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9629999999999997e-06 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.272793+0000", 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0093983079999999993 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171715 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266470+0000", 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017362790000000001, 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063979759999999997, 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171715 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266470+0000", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266470+0000", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266471+0000", 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:31:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266473+0000", 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266478+0000", 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5330000000000002e-06 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266482+0000", 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7050000000000001e-06 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171747 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270013+0000", 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9981937, 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029088819999999998, 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171747 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270013+0000", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270013+0000", 2024-09-12T09:31:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270013+0000", 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270016+0000", 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6860000000000002e-06 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270023+0000", 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8499999999999996e-06 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270030+0000", 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2110000000000001e-06 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171684 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263400+0000", 2024-09-12T09:31:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048062779999999998, 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.009579766, 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171684 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263400+0000", 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263400+0000", 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263401+0000", 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0699999999999997e-07 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263403+0000", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263409+0000", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.156e-06 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263414+0000", 2024-09-12T09:31:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8489999999999997e-06 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171716 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266487+0000", 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017194510000000001, 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065519000000000003, 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171716 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266487+0000", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266487+0000", 2024-09-12T09:31:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266488+0000", 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266489+0000", 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.407e-06 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266494+0000", 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1760000000000001e-06 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266498+0000", 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4800000000000001e-06 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171748 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270038+0000", 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9981685, 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030560299999999999, 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171748 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270038+0000", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270038+0000", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270039+0000", 2024-09-12T09:31:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-07 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270041+0000", 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2649999999999999e-06 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270047+0000", 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1539999999999999e-06 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270052+0000", 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4920000000000001e-06 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171685 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263420+0000", 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047868090000000004, 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097347570000000001, 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171685 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263420+0000", 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263420+0000", 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263420+0000", 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263422+0000", 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.784e-06 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263428+0000", 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8799999999999996e-06 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263432+0000", 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5560000000000001e-06 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171717 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266503+0000", 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017037879999999999, 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067138889999999998, 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171717 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266503+0000", 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266503+0000", 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266503+0000", 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3500000000000002e-07 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266505+0000", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.53e-06 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266511+0000", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2829999999999998e-06 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266518+0000", 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-09-12T09:31:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171749 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270060+0000", 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.998147, 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032167749999999998, 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171749 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270060+0000", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270060+0000", 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270060+0000", 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0200000000000001e-07 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270063+0000", 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0849999999999999e-06 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270069+0000", 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6830000000000002e-06 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270075+0000", 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6829999999999996e-06 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171686 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263439+0000", 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047673250000000002, 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099015330000000006, 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171686 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263439+0000", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263439+0000", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263440+0000", 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-09-12T09:31:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263442+0000", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263448+0000", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1689999999999997e-06 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263453+0000", 2024-09-12T09:31:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171718 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266544+0000", 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001663084, 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068557430000000001, 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171718 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266544+0000", 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266544+0000", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266544+0000", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6300000000000005e-07 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266548+0000", 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-09-12T09:31:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266558+0000", 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0774999999999999e-05 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266564+0000", 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0519999999999998e-06 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171750 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270092+0000", 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9981151, 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033706930000000001, 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171750 2024-09-12T09:31:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270092+0000", 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270092+0000", 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270092+0000", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6199999999999998e-07 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270099+0000", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2169999999999997e-06 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270106+0000", 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270113+0000", 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3459999999999999e-06 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171687 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263460+0000", 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047463460000000002, 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010062346, 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171687 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263460+0000", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263460+0000", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263461+0000", 2024-09-12T09:31:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263463+0000", 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.119e-06 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263469+0000", 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4479999999999999e-06 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263474+0000", 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8389999999999997e-06 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171719 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266597+0000", 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001609915, 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069879540000000002, 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171719 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266597+0000", 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266597+0000", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266597+0000", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6400000000000002e-07 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266600+0000", 2024-09-12T09:31:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.864e-06 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266613+0000", 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2363e-05 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266620+0000", 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6009999999999997e-06 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171751 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270121+0000", 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.998086, 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003524821, 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171751 2024-09-12T09:31:12.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270121+0000", 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270121+0000", 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270121+0000", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270123+0000", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1569999999999998e-06 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270129+0000", 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2790000000000004e-06 2024-09-12T09:31:12.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270135+0000", 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5709999999999997e-06 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171688 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263481+0000", 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047253900000000003, 2024-09-12T09:31:12.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010225071000000001, 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171688 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263481+0000", 2024-09-12T09:31:12.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263481+0000", 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263482+0000", 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263484+0000", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1409999999999999e-06 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263490+0000", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3110000000000002e-06 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263495+0000", 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0379999999999999e-06 2024-09-12T09:31:12.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171720 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266631+0000", 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015753780000000001, 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071402929999999998, 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171720 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266631+0000", 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266631+0000", 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266632+0000", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-07 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266635+0000", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8540000000000001e-06 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266643+0000", 2024-09-12T09:31:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4339999999999992e-06 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266650+0000", 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4110000000000001e-06 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171752 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270142+0000", 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.998065, 2024-09-12T09:31:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036887230000000001, 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171752 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270142+0000", 2024-09-12T09:31:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270142+0000", 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270142+0000", 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3499999999999996e-07 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270144+0000", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270151+0000", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270156+0000", 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2320000000000001e-06 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171689 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.263501+0000", 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047054050000000002, 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010397752, 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171689 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263501+0000", 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263501+0000", 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263502+0000", 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263504+0000", 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.933e-06 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263510+0000", 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0850000000000002e-06 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.263522+0000", 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2014999999999999e-05 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171721 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.266664+0000", 2024-09-12T09:31:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001542793, 2024-09-12T09:31:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072953619999999997, 2024-09-12T09:31:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171721 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266664+0000", 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266664+0000", 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266664+0000", 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0900000000000001e-07 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266667+0000", 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8380000000000002e-06 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266676+0000", 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4169999999999992e-06 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.266682+0000", 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5060000000000001e-06 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171753 3.14 3.7f3cd594 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:12.270163+0000", 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9980435, 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038596640000000001, 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171753 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270163+0000", 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270163+0000", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270163+0000", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270166+0000", 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.419e-06 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270172+0000", 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6630000000000004e-06 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.270177+0000", 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6550000000000003e-06 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:31:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 96 2024-09-12T09:31:12.627 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:12.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:12.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:12.629 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:12.627+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:12.729 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:12.728+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:12.788 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:12.797 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:12.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:12.797 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:12.797 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:12.797 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:12.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:12.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:12.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:12.838 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:12.856 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:12.855+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:12.857 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:12.876 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:12.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:12.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:12.921 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:12.921 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:12.931 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:12.928+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:12.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:13.032 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:13.031+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:13.084 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.084 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:13.133 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:13.132+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:13.197 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7] out_osds: [6, 1, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:13.198 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:13.198 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-12T09:31:13.198 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-12T09:31:13.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-12T09:31:13.233 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:13.231+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:13.258 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.258 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.271 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:13.356 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:13.355+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:13.439 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:13.436+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:13.444 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.444 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.458 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:13.511 INFO:teuthology.orchestra.run.smithi007.stdout:3657 write 0x99f95aa thru 0x9a06721 (0xd178 bytes) 2024-09-12T09:31:13.535 INFO:teuthology.orchestra.run.smithi007.stdout:3658 trunc from 0xc2261c3 to 0x4ee731a 2024-09-12T09:31:13.540 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:13.540+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:13.554 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:13.555 INFO:teuthology.orchestra.run.smithi007.stdout:3660 trunc from 0x4ee731a to 0x4a033ab 2024-09-12T09:31:13.581 INFO:teuthology.orchestra.run.smithi007.stdout:3661 write 0x80db2a8 thru 0x80e442d (0x9186 bytes) 2024-09-12T09:31:13.593 INFO:teuthology.orchestra.run.smithi007.stdout:3662 read 0x74a885d thru 0x74afed0 (0x7674 bytes) 2024-09-12T09:31:13.594 INFO:teuthology.orchestra.run.smithi007.stdout:3664 clone 36 order 24 su 524288 sc 12 2024-09-12T09:31:13.631 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.631 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.642 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:13.640+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:13.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:13.742 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:13.739+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:13.766 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.766 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:13.868 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:13.867+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:13.946 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:13.944+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:13.948 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:13.948 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:13.962 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:14.049 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:14.048+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:14.128 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.129 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:14.188 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:14.186+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-12T09:31:14.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:31:14.254 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:14.250+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:14.264 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.264 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:14.355 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:14.352+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:14.435 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-12T09:31:14.450 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.456 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:14.456+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:14.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:14.557 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:14.556+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:14.593 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.593 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:14.696 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:14.696+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:14.759 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:14.754+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:14.773 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.773 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:14.796 INFO:teuthology.orchestra.run.smithi007.stdout:3665 read 0x204caaa thru 0x20503f7 (0x394e bytes) 2024-09-12T09:31:14.817 INFO:teuthology.orchestra.run.smithi007.stdout:3666 write 0x82e4a29 thru 0x82e6b8a (0x2162 bytes) 2024-09-12T09:31:14.838 INFO:teuthology.orchestra.run.smithi007.stdout:3667 read 0x331e2d0 thru 0x332c1ad (0xdede bytes) 2024-09-12T09:31:14.838 INFO:teuthology.orchestra.run.smithi007.stdout:3668 read 0x7151e5d thru 0x7154997 (0x2b3b bytes) 2024-09-12T09:31:14.857 INFO:teuthology.orchestra.run.smithi007.stdout:3670 write 0xd236a40 thru 0xd241d43 (0xb304 bytes) 2024-09-12T09:31:14.860 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:14.860+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:14.864 INFO:teuthology.orchestra.run.smithi007.stdout:3671 punch from 0x18496ff to 0x18507f5, (0x70f6 bytes) 2024-09-12T09:31:14.864 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:14.865 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:14.865 INFO:teuthology.orchestra.run.smithi007.stdout:3674 read 0x8c258cd thru 0x8c282fd (0x2a31 bytes) 2024-09-12T09:31:14.865 INFO:teuthology.orchestra.run.smithi007.stdout:3675 read 0x887c713 thru 0x8886a95 (0xa383 bytes) 2024-09-12T09:31:14.866 INFO:teuthology.orchestra.run.smithi007.stdout:3677 read 0x14ca2aa thru 0x14d9b73 (0xf8ca bytes) 2024-09-12T09:31:14.866 INFO:teuthology.orchestra.run.smithi007.stdout:3678 punch from 0x2ad569a to 0x2addb08, (0x846e bytes) 2024-09-12T09:31:14.866 INFO:teuthology.orchestra.run.smithi007.stdout:3679 punch from 0x5ad214 to 0x5b2bdb, (0x59c7 bytes) 2024-09-12T09:31:14.866 INFO:teuthology.orchestra.run.smithi007.stdout:3681 write 0xc2adedf thru 0xc2b9fc8 (0xc0ea bytes) 2024-09-12T09:31:14.871 INFO:teuthology.orchestra.run.smithi007.stdout:3684 read 0x936b0c5 thru 0x936d908 (0x2844 bytes) 2024-09-12T09:31:14.872 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:14.872 INFO:teuthology.orchestra.run.smithi007.stdout:3687 writesame 0x83b1d45 thru 0x83b381e data_size 0x7(0x1ada bytes) 2024-09-12T09:31:14.886 INFO:teuthology.orchestra.run.smithi007.stdout:3688 punch from 0x7e03761 to 0x7e0b5aa, (0x7e49 bytes) 2024-09-12T09:31:14.886 INFO:teuthology.orchestra.run.smithi007.stdout:3689 writesame 0x78eca8a thru 0x78f11c8 data_size 0x17(0x473f bytes) 2024-09-12T09:31:14.895 INFO:teuthology.orchestra.run.smithi007.stdout:3691 read 0x227d198 thru 0x227ffe0 (0x2e49 bytes) 2024-09-12T09:31:14.898 INFO:teuthology.orchestra.run.smithi007.stdout:3692 trunc from 0xd241d44 to 0xbd1a720 2024-09-12T09:31:14.926 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:14.926 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:14.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:14.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:14.950 INFO:teuthology.orchestra.run.smithi007.stdout:3693 punch from 0x2dc5635 to 0x2dcc43d, (0x6e08 bytes) 2024-09-12T09:31:14.951 INFO:teuthology.orchestra.run.smithi007.stdout:3694 punch from 0x4ff94fe to 0x5004ac1, (0xb5c3 bytes) 2024-09-12T09:31:14.951 INFO:teuthology.orchestra.run.smithi007.stdout:3695 read 0x8841c56 thru 0x884b222 (0x95cd bytes) 2024-09-12T09:31:14.960 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:14.958+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:14.969 INFO:teuthology.orchestra.run.smithi007.stdout:3696 write 0xecf6275 thru 0xecfa683 (0x440f bytes) 2024-09-12T09:31:14.975 INFO:teuthology.orchestra.run.smithi007.stdout:3698 writesame 0x215aca thru 0x21adf7 data_size 0x75(0x532e bytes) 2024-09-12T09:31:14.986 INFO:teuthology.orchestra.run.smithi007.stdout:3699 punch from 0xb6328d4 to 0xb63ba54, (0x9180 bytes) 2024-09-12T09:31:14.986 INFO:teuthology.orchestra.run.smithi007.stdout:3700 read 0x2c49956 thru 0x2c511c3 (0x786e bytes) 2024-09-12T09:31:14.987 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.001 INFO:teuthology.orchestra.run.smithi007.stdout:3703 trunc from 0xecfa684 to 0x58720ea 2024-09-12T09:31:15.061 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:15.060+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:15.076 INFO:teuthology.orchestra.run.smithi007.stdout:3705 write 0xbcb9171 thru 0xbcb9495 (0x325 bytes) 2024-09-12T09:31:15.088 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:15.088 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:15.095 INFO:teuthology.orchestra.run.smithi007.stdout:3706 write 0xeaa05d8 thru 0xeaa3789 (0x31b2 bytes) 2024-09-12T09:31:15.101 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:15.101 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:15.102 INFO:teuthology.orchestra.run.smithi007.stdout:3708 read 0x122d959 thru 0x1237ccb (0xa373 bytes) 2024-09-12T09:31:15.103 INFO:teuthology.orchestra.run.smithi007.stdout:3709 punch from 0x1ca71ec to 0x1cac420, (0x5234 bytes) 2024-09-12T09:31:15.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:15.115 INFO:teuthology.orchestra.run.smithi007.stdout:3710 trunc from 0xeaa378a to 0x66b8814 2024-09-12T09:31:15.119 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:15.119 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:15.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:15.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:15.161 INFO:teuthology.orchestra.run.smithi007.stdout:3711 write 0x9a01e47 thru 0x9a0a005 (0x81bf bytes) 2024-09-12T09:31:15.161 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:15.160+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:15.164 INFO:teuthology.orchestra.run.smithi007.stdout:3712 write 0x16011a2 thru 0x160ba8a (0xa8e9 bytes) 2024-09-12T09:31:15.178 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:15.186 INFO:teuthology.orchestra.run.smithi007.stdout:3713 writesame 0xd75dee4 thru 0xd76c168 data_size 0x1(0xe285 bytes) 2024-09-12T09:31:15.186 INFO:teuthology.orchestra.run.smithi007.stdout:3714 write 0xd5f373b thru 0xd5fb859 (0x811f bytes) 2024-09-12T09:31:15.193 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.193 INFO:teuthology.orchestra.run.smithi007.stdout:3717 write 0x8fad79e thru 0x8faeabf (0x1322 bytes) 2024-09-12T09:31:15.195 INFO:teuthology.orchestra.run.smithi007.stdout:3718 writesame 0x8887fe5 thru 0x888e5af data_size 0x1(0x65cb bytes) 2024-09-12T09:31:15.201 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:15.222 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:15.222 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:15.231 INFO:teuthology.orchestra.run.smithi007.stdout:3719 writesame 0x9960310 thru 0x9965ec0 data_size 0x1(0x5bb1 bytes) 2024-09-12T09:31:15.237 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:15.237 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:15.241 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:31:15.249 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:15.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:15.261 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.261 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.262 INFO:teuthology.orchestra.run.smithi007.stdout:3722 read 0x8cc20bb thru 0x8cc60bb (0x4001 bytes) 2024-09-12T09:31:15.262 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:15.260+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:15.263 INFO:teuthology.orchestra.run.smithi007.stdout:3723 write 0xc262b10 thru 0xc26dc1d (0xb10e bytes) 2024-09-12T09:31:15.273 INFO:teuthology.orchestra.run.smithi007.stdout:3725 write 0x870ffdb thru 0x8719866 (0x988c bytes) 2024-09-12T09:31:15.293 INFO:teuthology.orchestra.run.smithi007.stdout:3726 write 0xe6948bb thru 0xe69fc97 (0xb3dd bytes) 2024-09-12T09:31:15.299 INFO:teuthology.orchestra.run.smithi007.stdout:3727 write 0x1ef1c21 thru 0x1ef7658 (0x5a38 bytes) 2024-09-12T09:31:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:3728 punch from 0x16b8c71 to 0x16c4e87, (0xc216 bytes) 2024-09-12T09:31:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:3729 write 0x69fbd2c thru 0x6a04096 (0x836b bytes) 2024-09-12T09:31:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.308 INFO:teuthology.orchestra.run.smithi007.stdout:3732 writesame 0x5f30b00 thru 0x5f39811 data_size 0x27(0x8d12 bytes) 2024-09-12T09:31:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:3733 trunc from 0xe69fc98 to 0xdd3f90f 2024-09-12T09:31:15.329 INFO:teuthology.orchestra.run.smithi007.stdout:3734 read 0x150d851 thru 0x151a08c (0xc83c bytes) 2024-09-12T09:31:15.343 INFO:teuthology.orchestra.run.smithi007.stdout:3736 write 0xe9560a5 thru 0xe961604 (0xb560 bytes) 2024-09-12T09:31:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:3738 punch from 0x83b4484 to 0x83bef2b, (0xaaa7 bytes) 2024-09-12T09:31:15.351 INFO:teuthology.orchestra.run.smithi007.stdout:3739 read 0xaf95a09 thru 0xaf98426 (0x2a1e bytes) 2024-09-12T09:31:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.352 INFO:teuthology.orchestra.run.smithi007.stdout:3741 write 0x25585d6 thru 0x25614a1 (0x8ecc bytes) 2024-09-12T09:31:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:3743 punch from 0xb5e57c7 to 0xb5ee27e, (0x8ab7 bytes) 2024-09-12T09:31:15.356 INFO:teuthology.orchestra.run.smithi007.stdout:3747 write 0x6c58c72 thru 0x6c5deb5 (0x5244 bytes) 2024-09-12T09:31:15.359 INFO:teuthology.orchestra.run.smithi007.stdout:3750 write 0x690d0e9 thru 0x6913f4d (0x6e65 bytes) 2024-09-12T09:31:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.363 INFO:teuthology.orchestra.run.smithi007.stdout:3753 write 0x461fbe1 thru 0x462367f (0x3a9f bytes) 2024-09-12T09:31:15.364 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:15.358+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:15.367 INFO:teuthology.orchestra.run.smithi007.stdout:3754 read 0x69b47d3 thru 0x69c23f6 (0xdc24 bytes) 2024-09-12T09:31:15.368 INFO:teuthology.orchestra.run.smithi007.stdout:3756 read 0x24211ab thru 0x2423458 (0x22ae bytes) 2024-09-12T09:31:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.369 INFO:teuthology.orchestra.run.smithi007.stdout:3758 writesame 0x439e76 thru 0x44641d data_size 0x73(0xc5a8 bytes) 2024-09-12T09:31:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:3759 punch from 0x6c82fa to 0x6d6e71, (0xeb77 bytes) 2024-09-12T09:31:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:3760 read 0x29bbb1 thru 0x2ab0f1 (0xf541 bytes) 2024-09-12T09:31:15.376 INFO:teuthology.orchestra.run.smithi007.stdout:3761 read 0xaf2b993 thru 0xaf3a230 (0xe89e bytes) 2024-09-12T09:31:15.377 INFO:teuthology.orchestra.run.smithi007.stdout:3762 read 0x8fcd664 thru 0x8fdd4f7 (0xfe94 bytes) 2024-09-12T09:31:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:3763 write 0x3f1bcf8 thru 0x3f236e6 (0x79ef bytes) 2024-09-12T09:31:15.382 INFO:teuthology.orchestra.run.smithi007.stdout:3764 writesame 0x1352848 thru 0x13562ac data_size 0x21(0x3a65 bytes) 2024-09-12T09:31:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.184072118000003, 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:31:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:31:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.169101388999998, 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:31:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.852254385000002, 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:31:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.852208026, 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:31:15.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.852100806000003, 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:15.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:31:15.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.852007315000002, 2024-09-12T09:31:15.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:31:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:31:15.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:15.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851940802000001, 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:31:15.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851879992999997, 2024-09-12T09:31:15.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:31:15.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851820701000001, 2024-09-12T09:31:15.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:31:15.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851760192, 2024-09-12T09:31:15.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:15.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:31:15.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.438 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851682136999997, 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.439 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:31:15.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.443 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851596762, 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:31:15.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851528442999999, 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:31:15.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851435858000002, 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:31:15.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851379948999998, 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:31:15.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:31:15.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851347625999999, 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:31:15.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:31:15.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851314033000001, 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:31:15.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851263091, 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:31:15.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:31:15.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851164936000004, 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:31:15.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.851107841999998, 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:31:15.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:31:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:31:15.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:31:15.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.492 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:15.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:15.494 INFO:teuthology.orchestra.run.smithi007.stdout:3765 trunc from 0xe961605 to 0x401c2f 2024-09-12T09:31:15.495 INFO:teuthology.orchestra.run.smithi007.stdout:3766 writesame 0x3e880ca thru 0x3e97b8f data_size 0x1(0xfac6 bytes) 2024-09-12T09:31:15.495 INFO:teuthology.orchestra.run.smithi007.stdout:3768 read 0x31cd264 thru 0x31da6da (0xd477 bytes) 2024-09-12T09:31:15.495 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:15.492+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:15.512 INFO:teuthology.orchestra.run.smithi007.stdout:3771 trunc from 0x3e97b90 to 0xaea8351 2024-09-12T09:31:15.529 INFO:teuthology.orchestra.run.smithi007.stdout:3772 read 0x54230ad thru 0x542b48d (0x83e1 bytes) 2024-09-12T09:31:15.530 INFO:teuthology.orchestra.run.smithi007.stdout:3773 read 0x3235cd8 thru 0x3240688 (0xa9b1 bytes) 2024-09-12T09:31:15.531 INFO:teuthology.orchestra.run.smithi007.stdout:3775 read 0xf35b22 thru 0xf39154 (0x3633 bytes) 2024-09-12T09:31:15.532 INFO:teuthology.orchestra.run.smithi007.stdout:3777 writesame 0xab9a37c thru 0xaba2f82 data_size 0x9(0x8c07 bytes) 2024-09-12T09:31:15.538 INFO:teuthology.orchestra.run.smithi007.stdout:3778 writesame 0x57a5940 thru 0x57abe0d data_size 0x17(0x64ce bytes) 2024-09-12T09:31:15.543 INFO:teuthology.orchestra.run.smithi007.stdout:3779 punch from 0x38b78ef to 0x38bdd5d, (0x646e bytes) 2024-09-12T09:31:15.543 INFO:teuthology.orchestra.run.smithi007.stdout:3781 read 0x7b3fbf1 thru 0x7b46a64 (0x6e74 bytes) 2024-09-12T09:31:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:3782 read 0x72d3f6a thru 0x72dbed1 (0x7f68 bytes) 2024-09-12T09:31:15.562 INFO:teuthology.orchestra.run.smithi007.stdout:3784 writesame 0xd90e444 thru 0xd91acbc data_size 0x1(0xc879 bytes) 2024-09-12T09:31:15.562 INFO:teuthology.orchestra.run.smithi007.stdout:3785 writesame 0xb2dfe27 thru 0xb2ebb62 data_size 0x42(0xbd3c bytes) 2024-09-12T09:31:15.571 INFO:teuthology.orchestra.run.smithi007.stdout:3786 read 0x381ecb7 thru 0x3829040 (0xa38a bytes) 2024-09-12T09:31:15.572 INFO:teuthology.orchestra.run.smithi007.stdout:3787 punch from 0xd2890ea to 0xd294af0, (0xba06 bytes) 2024-09-12T09:31:15.572 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:3791 punch from 0x76667a7 to 0x7671bfd, (0xb456 bytes) 2024-09-12T09:31:15.577 INFO:teuthology.orchestra.run.smithi007.stdout:3792 trunc from 0xd91acbd to 0xad6cce1 2024-09-12T09:31:15.595 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:15.590+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:15.600 INFO:teuthology.orchestra.run.smithi007.stdout:3793 read 0x493b92c thru 0x49494c7 (0xdb9c bytes) 2024-09-12T09:31:15.601 INFO:teuthology.orchestra.run.smithi007.stdout:3795 read 0x5fa5ac4 thru 0x5fa81bc (0x26f9 bytes) 2024-09-12T09:31:15.601 INFO:teuthology.orchestra.run.smithi007.stdout:3796 write 0x24189e3 thru 0x2422418 (0x9a36 bytes) 2024-09-12T09:31:15.605 INFO:teuthology.orchestra.run.smithi007.stdout:3798 read 0x5996baf thru 0x5997c0e (0x1060 bytes) 2024-09-12T09:31:15.605 INFO:teuthology.orchestra.run.smithi007.stdout:3799 writesame 0x3dfac1e thru 0x3e05704 data_size 0x1(0xaae7 bytes) 2024-09-12T09:31:15.637 INFO:teuthology.orchestra.run.smithi007.stdout:3800 write 0x185ca57 thru 0x186ac1c (0xe1c6 bytes) 2024-09-12T09:31:15.641 INFO:teuthology.orchestra.run.smithi007.stdout:3801 writesame 0xa6a35fe thru 0xa6a5897 data_size 0x2(0x229a bytes) 2024-09-12T09:31:15.648 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:15.649 INFO:teuthology.orchestra.run.smithi007.stdout:3803 read 0x5f98fc thru 0x600e43 (0x7548 bytes) 2024-09-12T09:31:15.650 INFO:teuthology.orchestra.run.smithi007.stdout:3804 punch from 0x9ee6fd4 to 0x9ef3695, (0xc6c1 bytes) 2024-09-12T09:31:15.650 INFO:teuthology.orchestra.run.smithi007.stdout:3806 punch from 0x29da8d8 to 0x29df9be, (0x50e6 bytes) 2024-09-12T09:31:15.650 INFO:teuthology.orchestra.run.smithi007.stdout:3807 read 0x22ae558 thru 0x22b8d8f (0xa838 bytes) 2024-09-12T09:31:15.651 INFO:teuthology.orchestra.run.smithi007.stdout:3808 read 0x646e3ef thru 0x647a666 (0xc278 bytes) 2024-09-12T09:31:15.652 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:15.652 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:15.653 INFO:teuthology.orchestra.run.smithi007.stdout:3810 writesame 0x9766423 thru 0x9774321 data_size 0x3(0xdeff bytes) 2024-09-12T09:31:15.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:15.677 INFO:teuthology.orchestra.run.smithi007.stdout:3811 write 0x8521e43 thru 0x8531940 (0xfafe bytes) 2024-09-12T09:31:15.681 INFO:teuthology.orchestra.run.smithi007.stdout:3812 read 0x99ebcbd thru 0x99f263f (0x6983 bytes) 2024-09-12T09:31:15.687 INFO:teuthology.orchestra.run.smithi007.stdout:3813 trunc from 0xad6cce1 to 0xe10ce84 2024-09-12T09:31:15.696 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:15.696+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:15.700 INFO:teuthology.orchestra.run.smithi007.stdout:3814 writesame 0x4ef0b5d thru 0x4ef5635 data_size 0x3(0x4ad9 bytes) 2024-09-12T09:31:15.709 INFO:teuthology.orchestra.run.smithi007.stdout:3815 write 0xa7cd615 thru 0xa7d8321 (0xad0d bytes) 2024-09-12T09:31:15.717 INFO:teuthology.orchestra.run.smithi007.stdout:3818 write 0x3e557e1 thru 0x3e56363 (0xb83 bytes) 2024-09-12T09:31:15.721 INFO:teuthology.orchestra.run.smithi007.stdout:3819 writesame 0xce74490 thru 0xce82ab9 data_size 0x11(0xe62a bytes) 2024-09-12T09:31:15.731 INFO:teuthology.orchestra.run.smithi007.stdout:3820 punch from 0xa409546 to 0xa4171c8, (0xdc82 bytes) 2024-09-12T09:31:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:3821 trunc from 0xe10ce84 to 0x467408 2024-09-12T09:31:15.796 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:15.796+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:15.807 INFO:teuthology.orchestra.run.smithi007.stdout:3822 trunc from 0x467408 to 0xb090ed 2024-09-12T09:31:15.812 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:15.812 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:15.812 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:15.812 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:15.812 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:15.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:15.846 INFO:teuthology.orchestra.run.smithi007.stdout:3824 write 0x67c2a9a thru 0x67c71fc (0x4763 bytes) 2024-09-12T09:31:15.849 INFO:teuthology.orchestra.run.smithi007.stdout:3826 punch from 0xcc7a1d to 0xcd749e, (0xfa81 bytes) 2024-09-12T09:31:15.849 INFO:teuthology.orchestra.run.smithi007.stdout:3827 punch from 0x2a21d3 to 0x2a8344, (0x6171 bytes) 2024-09-12T09:31:15.849 INFO:teuthology.orchestra.run.smithi007.stdout:3828 writesame 0x3dd9352 thru 0x3de1fee data_size 0x47(0x8c9d bytes) 2024-09-12T09:31:15.853 INFO:teuthology.orchestra.run.smithi007.stdout:3829 write 0xfce291 thru 0xfd58de (0x764e bytes) 2024-09-12T09:31:15.879 INFO:teuthology.orchestra.run.smithi007.stdout:3830 write 0xa843f71 thru 0xa848d25 (0x4db5 bytes) 2024-09-12T09:31:15.881 INFO:teuthology.orchestra.run.smithi007.stdout:3831 clone 37 order 23 su 2097152 sc 12 2024-09-12T09:31:15.897 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:15.894+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.668127386999998, 2024-09-12T09:31:15.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:15.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:15.974 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.485708469000002, 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:15.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:15.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.092921955999998, 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:15.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.745392068000001, 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:15.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:15.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.742938158000001, 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:15.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.740828270999998, 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:15.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:15.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.245081626000001, 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:15.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:15.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.214673340000001, 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:16.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.993708780999999, 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:16.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:16.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:16.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.294597498, 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:16.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:16.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.294580569000001, 2024-09-12T09:31:16.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:16.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.027 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.255818971, 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:16.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:16.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.198450482, 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:16.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.198427063, 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.167198058, 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.086095101, 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.086069106, 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:16.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.832852618, 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:16.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:16.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.610168542, 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:16.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:16.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.610141734000001, 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:16.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.084 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:16.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:16.086 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:16.082+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:16.186 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:16.184+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:16.267 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:16.267 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:16.267 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:16.267 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:16.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:16.287 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:16.282+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:16.387 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone36 from 0xa848d26 (overlap 0x401c2f) to 0x26774f 2024-09-12T09:31:16.388 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:16.388+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:16.460 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:16.460 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:16.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:16.489 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:16.486+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:16.521 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-12T09:31:16.590 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:16.588+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:16.657 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148930 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 937984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.220887435000002, 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097427275999999993, 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148930 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435664+0000", 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435666+0000", 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435672+0000", 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4969999999999998e-06 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435677+0000", 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1780000000000002e-06 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532783+0000", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097105891 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532861+0000", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7760000000000001e-05 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533090+0000", 2024-09-12T09:31:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022925500000000001 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.220866259000001, 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.220841027999999, 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.220803107999998, 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.220773106999999, 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:16.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:16.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.212738686000002, 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:16.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:16.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.212417812999998, 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:16.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211993987, 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211942508, 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211913760000002, 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211363294000002, 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211320472000001, 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.211274581000001, 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.210763022999998, 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.210715243999999, 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.210659597999999, 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.210627577, 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.210063428000002, 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.209669526999999, 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.209277829000001, 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:16.755 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:16.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:16.757 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:16.756+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:16.820 INFO:teuthology.orchestra.run.smithi007.stdout:3832 punch from 0x1624574 to 0x1628123, (0x3baf bytes) 2024-09-12T09:31:16.828 INFO:teuthology.orchestra.run.smithi007.stdout:3834 write 0x4f26930 thru 0x4f2d580 (0x6c51 bytes) 2024-09-12T09:31:16.838 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:16.839 INFO:teuthology.orchestra.run.smithi007.stdout:3838 write 0x2fbfbf6 thru 0x2fc754b (0x7956 bytes) 2024-09-12T09:31:16.848 INFO:teuthology.orchestra.run.smithi007.stdout:3840 write 0x6a07a3c thru 0x6a127e6 (0xadab bytes) 2024-09-12T09:31:16.857 INFO:teuthology.orchestra.run.smithi007.stdout:3841 punch from 0x9ef4d1c to 0x9f02827, (0xdb0b bytes) 2024-09-12T09:31:16.858 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:16.856+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:16.858 INFO:teuthology.orchestra.run.smithi007.stdout:3842 writesame 0x4cb7f97 thru 0x4cbd0be data_size 0x38(0x5128 bytes) 2024-09-12T09:31:16.867 INFO:teuthology.orchestra.run.smithi007.stdout:3843 punch from 0x3892877 to 0x389816c, (0x58f5 bytes) 2024-09-12T09:31:16.867 INFO:teuthology.orchestra.run.smithi007.stdout:3844 punch from 0x9f87ea7 to 0x9f8d553, (0x56ac bytes) 2024-09-12T09:31:16.867 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:16.880 INFO:teuthology.orchestra.run.smithi007.stdout:3846 write 0xaa74035 thru 0xaa7eb2b (0xaaf7 bytes) 2024-09-12T09:31:16.894 INFO:teuthology.orchestra.run.smithi007.stdout:3847 trunc from 0xaa7eb2c to 0xc0fe1e6 2024-09-12T09:31:16.894 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:16.895 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:16.895 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:16.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:16.924 INFO:teuthology.orchestra.run.smithi007.stdout:3848 writesame 0xc4146a4 thru 0xc4186e9 data_size 0x13(0x4046 bytes) 2024-09-12T09:31:16.925 INFO:teuthology.orchestra.run.smithi007.stdout:3850 write 0x37816c7 thru 0x3785c7f (0x45b9 bytes) 2024-09-12T09:31:16.929 INFO:teuthology.orchestra.run.smithi007.stdout:3852 write 0x8ce0b54 thru 0x8cea836 (0x9ce3 bytes) 2024-09-12T09:31:16.952 INFO:teuthology.orchestra.run.smithi007.stdout:3853 write 0xcd37375 thru 0xcd43df1 (0xca7d bytes) 2024-09-12T09:31:16.958 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:16.954+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:16.959 INFO:teuthology.orchestra.run.smithi007.stdout:3854 write 0x4b3440f thru 0x4b3604b (0x1c3d bytes) 2024-09-12T09:31:16.979 INFO:teuthology.orchestra.run.smithi007.stdout:3855 write 0xcbfbe98 thru 0xcc04484 (0x85ed bytes) 2024-09-12T09:31:16.986 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:16.986 INFO:teuthology.orchestra.run.smithi007.stdout:3858 writesame 0x14fc711 thru 0x14ff237 data_size 0x1(0x2b27 bytes) 2024-09-12T09:31:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:3860 punch from 0x6b7d50a to 0x6b84c4a, (0x7740 bytes) 2024-09-12T09:31:17.000 INFO:teuthology.orchestra.run.smithi007.stdout:3861 writesame 0x7056b4 thru 0x713cd4 data_size 0x1(0xe621 bytes) 2024-09-12T09:31:17.032 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:17.033 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:17.041 INFO:teuthology.orchestra.run.smithi007.stdout:3862 write 0x10ad348 thru 0x10b19b5 (0x466e bytes) 2024-09-12T09:31:17.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:17.060 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:17.060+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:17.068 INFO:teuthology.orchestra.run.smithi007.stdout:3863 writesame 0xed9293a thru 0xeda1aef data_size 0x2(0xf1b6 bytes) 2024-09-12T09:31:17.069 INFO:teuthology.orchestra.run.smithi007.stdout:3864 read 0x16f147c thru 0x16f5037 (0x3bbc bytes) 2024-09-12T09:31:17.070 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.070 INFO:teuthology.orchestra.run.smithi007.stdout:3866 writesame 0x64bdf76 thru 0x64c82ed data_size 0x8(0xa378 bytes) 2024-09-12T09:31:17.083 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.083 INFO:teuthology.orchestra.run.smithi007.stdout:3869 punch from 0x2689389 to 0x269615a, (0xcdd1 bytes) 2024-09-12T09:31:17.083 INFO:teuthology.orchestra.run.smithi007.stdout:3870 read 0x20f8141 thru 0x20fc5ae (0x446e bytes) 2024-09-12T09:31:17.084 INFO:teuthology.orchestra.run.smithi007.stdout:3871 trunc from 0xeda1af0 to 0xeb3d912 2024-09-12T09:31:17.099 INFO:teuthology.orchestra.run.smithi007.stdout:3872 punch from 0xda702a4 to 0xda709df, (0x73b bytes) 2024-09-12T09:31:17.099 INFO:teuthology.orchestra.run.smithi007.stdout:3873 write 0xeafde27 thru 0xeb0095d (0x2b37 bytes) 2024-09-12T09:31:17.107 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.107 INFO:teuthology.orchestra.run.smithi007.stdout:3876 write 0x3a47af9 thru 0x3a54bf2 (0xd0fa bytes) 2024-09-12T09:31:17.116 INFO:teuthology.orchestra.run.smithi007.stdout:3880 writesame 0xad304e3 thru 0xad31e59 data_size 0x29(0x1977 bytes) 2024-09-12T09:31:17.119 INFO:teuthology.orchestra.run.smithi007.stdout:3881 read 0x1b716a1 thru 0x1b7e36f (0xcccf bytes) 2024-09-12T09:31:17.120 INFO:teuthology.orchestra.run.smithi007.stdout:3882 writesame 0x6dcfc9d thru 0x6dd29d5 data_size 0x11(0x2d39 bytes) 2024-09-12T09:31:17.128 INFO:teuthology.orchestra.run.smithi007.stdout:3883 read 0x5040680 thru 0x5047553 (0x6ed4 bytes) 2024-09-12T09:31:17.135 INFO:teuthology.orchestra.run.smithi007.stdout:3884 trunc from 0xeb3d912 to 0xcd6f7a0 2024-09-12T09:31:17.156 INFO:teuthology.orchestra.run.smithi007.stdout:3888 punch from 0x21ab91a to 0x21ac4e1, (0xbc7 bytes) 2024-09-12T09:31:17.156 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.157 INFO:teuthology.orchestra.run.smithi007.stdout:3890 read 0xe13d0c thru 0xe1a40c (0x6701 bytes) 2024-09-12T09:31:17.161 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:17.158+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:17.186 INFO:teuthology.orchestra.run.smithi007.stdout:3892 trunc from 0xcd6f7a0 to 0x2335f62 2024-09-12T09:31:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:3895 punch from 0x4a5381 to 0x4b2676, (0xd2f5 bytes) 2024-09-12T09:31:17.236 INFO:teuthology.orchestra.run.smithi007.stdout:3896 read 0xa509e1 thru 0xa58305 (0x7925 bytes) 2024-09-12T09:31:17.238 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.947663460000001, 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.488167231000006, 2024-09-12T09:31:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.114750279999996, 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.114703032999998, 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.114409957000007, 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.113848024999996, 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.113796661999999, 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.113223782000006, 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.113071482999999, 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.112714393000005, 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.112089423, 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.112038087000002, 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.111243297000001, 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.111168406999994, 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.111116086999999, 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.111067489000007, 2024-09-12T09:31:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.110813023999995, 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.107358734000002, 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.049790186999999, 2024-09-12T09:31:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.578518950000003, 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:17.335 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:17.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:31:17.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:17.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:3897 writesame 0x5825205 thru 0x5825e47 data_size 0x1(0xc43 bytes) 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:3904 write 0xd360687 thru 0xd366559 (0x5ed3 bytes) 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:3908 read 0x26f1656 thru 0x26f9a77 (0x8422 bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3910 writesame 0x420a3c4 thru 0x420c21f data_size 0x1d(0x1e5c bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3914 write 0xcdd6750 thru 0xcddff9c (0x984d bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3915 write 0x4bbf7de thru 0x4bce243 (0xea66 bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3916 write 0x731eb3 thru 0x73ad03 (0x8e51 bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3918 writesame 0xabbf6b8 thru 0xabce09f data_size 0x28(0xe9e8 bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3919 read 0xc9569ff thru 0xc95a7d4 (0x3dd6 bytes) 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.340 INFO:teuthology.orchestra.run.smithi007.stdout:3921 trunc from 0xd36655a to 0xd74c41b 2024-09-12T09:31:17.341 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:17.336+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:17.348 INFO:teuthology.orchestra.run.smithi007.stdout:3922 write 0x9d38926 thru 0x9d46715 (0xddf0 bytes) 2024-09-12T09:31:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:3923 trunc from 0xd74c41b to 0x31588f0 2024-09-12T09:31:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:3925 read 0x2e1bc3d thru 0x2e214b3 (0x5877 bytes) 2024-09-12T09:31:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:3928 read 0x11273b9 thru 0x11323f6 (0xb03e bytes) 2024-09-12T09:31:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:3930 write 0x332976d thru 0x3330765 (0x6ff9 bytes) 2024-09-12T09:31:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:3934 read 0x19e9773 thru 0x19ec6e7 (0x2f75 bytes) 2024-09-12T09:31:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:3935 trunc from 0x3330766 to 0xa5d762 2024-09-12T09:31:17.437 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:17.436+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:3936 punch from 0x162847 to 0x1692f5, (0x6aae bytes) 2024-09-12T09:31:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:3937 read 0x7fb008 thru 0x803e03 (0x8dfc bytes) 2024-09-12T09:31:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:3939 writesame 0x41b59c6 thru 0x41c513d data_size 0x4(0xf778 bytes) 2024-09-12T09:31:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:3940 write 0x5622250 thru 0x562fede (0xdc8f bytes) 2024-09-12T09:31:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:3941 read 0x315e7f thru 0x31b44f (0x55d1 bytes) 2024-09-12T09:31:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:3942 read 0x42275a8 thru 0x42288ce (0x1327 bytes) 2024-09-12T09:31:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:3943 punch from 0x3148faa to 0x314fd93, (0x6de9 bytes) 2024-09-12T09:31:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:3944 trunc from 0x562fedf to 0xeaab5d5 2024-09-12T09:31:17.510 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:17.510 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:17.512 INFO:teuthology.orchestra.run.smithi007.stdout:3945 write 0xb1e8f4e thru 0xb1ea8fc (0x19af bytes) 2024-09-12T09:31:17.514 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:17.514 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:17.515 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:17.515 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:17.517 INFO:teuthology.orchestra.run.smithi007.stdout:3946 read 0xbcd2208 thru 0xbcda19f (0x7f98 bytes) 2024-09-12T09:31:17.517 INFO:teuthology.orchestra.run.smithi007.stdout:3947 read 0x79fd9d8 thru 0x79fecf3 (0x131c bytes) 2024-09-12T09:31:17.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:17.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:17.534 INFO:teuthology.orchestra.run.smithi007.stdout:3948 trunc from 0xeaab5d5 to 0x3077d38 2024-09-12T09:31:17.551 INFO:teuthology.orchestra.run.smithi007.stdout:3949 writesame 0x2d7baa0 thru 0x2d8a8c1 data_size 0xa3(0xee22 bytes) 2024-09-12T09:31:17.554 INFO:teuthology.orchestra.run.smithi007.stdout:3950 read 0x1dca84a thru 0x1dd0321 (0x5ad8 bytes) 2024-09-12T09:31:17.563 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:17.567 INFO:teuthology.orchestra.run.smithi007.stdout:3951 trunc from 0x3077d38 to 0xb85750d 2024-09-12T09:31:17.573 INFO:teuthology.orchestra.run.smithi007.stdout:3954 write 0x28e82d9 thru 0x28ef03e (0x6d66 bytes) 2024-09-12T09:31:17.576 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:17.574+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:17.586 INFO:teuthology.orchestra.run.smithi007.stdout:3955 trunc from 0xb85750d to 0x4ba2a19 2024-09-12T09:31:17.596 INFO:teuthology.orchestra.run.smithi007.stdout:3956 read 0x1a7ad8 thru 0x1b26c7 (0xabf0 bytes) 2024-09-12T09:31:17.597 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:17.598 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:17.601 INFO:teuthology.orchestra.run.smithi007.stdout:3957 trunc from 0x4ba2a19 to 0x23c2af5 2024-09-12T09:31:17.612 INFO:teuthology.orchestra.run.smithi007.stdout:3959 read 0x1d3d97 thru 0x1e1e27 (0xe091 bytes) 2024-09-12T09:31:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:17.629 INFO:teuthology.orchestra.run.smithi007.stdout:3960 trunc from 0x23c2af5 to 0xc1aad1d 2024-09-12T09:31:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:3961 write 0xa3f5a5 thru 0xa4b239 (0xbc95 bytes) 2024-09-12T09:31:17.638 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:17.636+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:17.640 INFO:teuthology.orchestra.run.smithi007.stdout:3962 read 0x7c9dee3 thru 0x7c9ed12 (0xe30 bytes) 2024-09-12T09:31:17.642 INFO:teuthology.orchestra.run.smithi007.stdout:3963 trunc from 0xc1aad1d to 0xa5825a1 2024-09-12T09:31:17.650 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:17.650 INFO:teuthology.orchestra.run.smithi007.stdout:3964 read 0x92a5f7 thru 0x92d1d7 (0x2be1 bytes) 2024-09-12T09:31:17.651 INFO:teuthology.orchestra.run.smithi007.stdout:3965 punch from 0x8f325bf to 0x8f393f4, (0x6e35 bytes) 2024-09-12T09:31:17.651 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.655 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:17.656 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:17.660 INFO:teuthology.orchestra.run.smithi007.stdout:3967 write 0xb550f36 thru 0xb559b81 (0x8c4c bytes) 2024-09-12T09:31:17.666 INFO:teuthology.orchestra.run.smithi007.stdout:3968 trunc from 0xb559b82 to 0xaf38862 2024-09-12T09:31:17.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:17.675 INFO:teuthology.orchestra.run.smithi007.stdout:3969 writesame 0x12fd0b2 thru 0x12ff7ce data_size 0x13(0x271d bytes) 2024-09-12T09:31:17.679 INFO:teuthology.orchestra.run.smithi007.stdout:3970 write 0x6840b04 thru 0x684b72f (0xac2c bytes) 2024-09-12T09:31:17.683 INFO:teuthology.orchestra.run.smithi007.stdout:3972 read 0x9a942f2 thru 0x9a9861e (0x432d bytes) 2024-09-12T09:31:17.683 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:17.683 INFO:teuthology.orchestra.run.smithi007.stdout:3974 writesame 0x11644ed thru 0x116c2f2 data_size 0x39(0x7e06 bytes) 2024-09-12T09:31:17.685 INFO:teuthology.orchestra.run.smithi007.stdout:3975 write 0x7d94d09 thru 0x7d98180 (0x3478 bytes) 2024-09-12T09:31:17.686 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:17.686 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:17.698 INFO:teuthology.orchestra.run.smithi007.stdout:3976 trunc from 0xaf38862 to 0x44be7f4 2024-09-12T09:31:17.704 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:17.720 INFO:teuthology.orchestra.run.smithi007.stdout:3977 read 0x225945b thru 0x2263ac1 (0xa667 bytes) 2024-09-12T09:31:17.721 INFO:teuthology.orchestra.run.smithi007.stdout:3978 read 0x37fec5f thru 0x3802f88 (0x432a bytes) 2024-09-12T09:31:17.731 INFO:teuthology.orchestra.run.smithi007.stdout:3980 writesame 0x5045c75 thru 0x5050909 data_size 0x3(0xac95 bytes) 2024-09-12T09:31:17.733 INFO:teuthology.orchestra.run.smithi007.stdout:3981 punch from 0x214abc4 to 0x21507d6, (0x5c12 bytes) 2024-09-12T09:31:17.733 INFO:teuthology.orchestra.run.smithi007.stdout:3982 read 0x20f2a7c thru 0x210035f (0xd8e4 bytes) 2024-09-12T09:31:17.743 INFO:teuthology.orchestra.run.smithi007.stdout:3984 writesame 0x5d4610a thru 0x5d55e14 data_size 0x27(0xfd0b bytes) 2024-09-12T09:31:17.756 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:17.754+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:17.763 INFO:teuthology.orchestra.run.smithi007.stdout:3986 writesame 0xc1d9363 thru 0xc1e4a69 data_size 0x1(0xb707 bytes) 2024-09-12T09:31:17.763 INFO:teuthology.orchestra.run.smithi007.stdout:3987 write 0x3dd87fb thru 0x3dded50 (0x6556 bytes) 2024-09-12T09:31:17.767 INFO:teuthology.orchestra.run.smithi007.stdout:3988 read 0x1a76b48 thru 0x1a7e308 (0x77c1 bytes) 2024-09-12T09:31:17.767 INFO:teuthology.orchestra.run.smithi007.stdout:3989 read 0x9c53e4a thru 0x9c5a541 (0x66f8 bytes) 2024-09-12T09:31:17.767 INFO:teuthology.orchestra.run.smithi007.stdout:3990 read 0x2893c3e thru 0x289bc52 (0x8015 bytes) 2024-09-12T09:31:17.778 INFO:teuthology.orchestra.run.smithi007.stdout:3991 trunc from 0xc1e4a6a to 0x5948f85 2024-09-12T09:31:17.789 INFO:teuthology.orchestra.run.smithi007.stdout:3993 clone 38 order 20 su 131072 sc 12 2024-09-12T09:31:17.835 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:17.835 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:17.840 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:17.838+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:17.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:17.941 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:17.938+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:18.016 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.017 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.031 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:18.072 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:18.070+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:18.143 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:18.140+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:18.196 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.196 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:18.260 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:18.258+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:31:18.337 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.337 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.345 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:18.344+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:18.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone37 from 0x5948f85 (overlap 0xa5d762) to 0x109589 2024-09-12T09:31:18.446 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:18.442+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:18.526 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.526 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:18.547 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:18.544+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:18.577 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-12T09:31:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:3994 read 0xa9a22b thru 0xaa0dac (0x6b82 bytes) 2024-09-12T09:31:18.629 INFO:teuthology.orchestra.run.smithi007.stdout:3995 write 0xa093cb9 thru 0xa099005 (0x534d bytes) 2024-09-12T09:31:18.646 INFO:teuthology.orchestra.run.smithi007.stdout:3997 trunc from 0xa099006 to 0xcc70a31 2024-09-12T09:31:18.648 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:18.648+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:18.661 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:18.679 INFO:teuthology.orchestra.run.smithi007.stdout:3999 write 0xddd5a7c thru 0xddd6ddd (0x1362 bytes) 2024-09-12T09:31:18.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:18.686 INFO:teuthology.orchestra.run.smithi007.stdout:4002 read 0x7c9cbd9 thru 0x7ca429d (0x76c5 bytes) 2024-09-12T09:31:18.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:18.686 INFO:teuthology.orchestra.run.smithi007.stdout:4004 write 0xd0bca71 thru 0xd0bd322 (0x8b2 bytes) 2024-09-12T09:31:18.693 INFO:teuthology.orchestra.run.smithi007.stdout:4005 writesame 0x8f1ba48 thru 0x8f27143 data_size 0x62(0xb6fc bytes) 2024-09-12T09:31:18.715 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.715 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:18.735 INFO:teuthology.orchestra.run.smithi007.stdout:4006 trunc from 0xddd6dde to 0x1c69755 2024-09-12T09:31:18.776 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:18.774+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:18.784 INFO:teuthology.orchestra.run.smithi007.stdout:4007 read 0xf693ab thru 0xf76caa (0xd900 bytes) 2024-09-12T09:31:18.804 INFO:teuthology.orchestra.run.smithi007.stdout:4008 trunc from 0x1c69755 to 0x8d846bd 2024-09-12T09:31:18.818 INFO:teuthology.orchestra.run.smithi007.stdout:4009 writesame 0x7bda00e thru 0x7be7c92 data_size 0x1(0xdc85 bytes) 2024-09-12T09:31:18.850 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:18.846+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:18.863 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:18.863 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:18.865 INFO:teuthology.orchestra.run.smithi007.stdout:4010 writesame 0x60c0c9d thru 0x60c806f data_size 0x1(0x73d3 bytes) 2024-09-12T09:31:18.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:18.911 INFO:teuthology.orchestra.run.smithi007.stdout:4011 writesame 0xba8c42c thru 0xba98434 data_size 0x7(0xc009 bytes) 2024-09-12T09:31:18.911 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:18.911 INFO:teuthology.orchestra.run.smithi007.stdout:4013 read 0x1862664 thru 0x186c418 (0x9db5 bytes) 2024-09-12T09:31:18.911 INFO:teuthology.orchestra.run.smithi007.stdout:4014 write 0xa3fd34d thru 0xa3fd729 (0x3dd bytes) 2024-09-12T09:31:18.917 INFO:teuthology.orchestra.run.smithi007.stdout:4015 punch from 0x91d47a to 0x92059e, (0x3124 bytes) 2024-09-12T09:31:18.918 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:18.918 INFO:teuthology.orchestra.run.smithi007.stdout:4017 punch from 0x5838a86 to 0x583e37f, (0x58f9 bytes) 2024-09-12T09:31:18.938 INFO:teuthology.orchestra.run.smithi007.stdout:4021 trunc from 0xba98435 to 0x701b6 2024-09-12T09:31:18.951 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:18.948+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:18.990 INFO:teuthology.orchestra.run.smithi007.stdout:4022 trunc from 0x701b6 to 0x64cd8ed 2024-09-12T09:31:19.000 INFO:teuthology.orchestra.run.smithi007.stdout:4024 read 0x31bd47d thru 0x31beac5 (0x1649 bytes) 2024-09-12T09:31:19.013 INFO:teuthology.orchestra.run.smithi007.stdout:4026 write 0x7c6b6f4 thru 0x7c74daf (0x96bc bytes) 2024-09-12T09:31:19.022 INFO:teuthology.orchestra.run.smithi007.stdout:4027 writesame 0x1f7a5af thru 0x1f83413 data_size 0x1d(0x8e65 bytes) 2024-09-12T09:31:19.029 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.029 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.031 INFO:teuthology.orchestra.run.smithi007.stdout:4028 read 0x34a70f0 thru 0x34ae391 (0x72a2 bytes) 2024-09-12T09:31:19.032 INFO:teuthology.orchestra.run.smithi007.stdout:4029 write 0x68aaaf2 thru 0x68addfc (0x330b bytes) 2024-09-12T09:31:19.041 INFO:teuthology.orchestra.run.smithi007.stdout:4030 trunc from 0x7c74db0 to 0x9e96fa2 2024-09-12T09:31:19.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:19.051 INFO:teuthology.orchestra.run.smithi007.stdout:4032 read 0x666ff9a thru 0x667919d (0x9204 bytes) 2024-09-12T09:31:19.051 INFO:teuthology.orchestra.run.smithi007.stdout:4034 read 0x815be6f thru 0x8165b28 (0x9cba bytes) 2024-09-12T09:31:19.051 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:19.051 INFO:teuthology.orchestra.run.smithi007.stdout:4039 write 0x3378520 thru 0x337cda5 (0x4886 bytes) 2024-09-12T09:31:19.052 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:19.052+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:19.060 INFO:teuthology.orchestra.run.smithi007.stdout:4040 trunc from 0x9e96fa2 to 0xc61f929 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4042 read 0x785c564 thru 0x786561e (0x90bb bytes) 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4043 punch from 0xb1617a9 to 0xb16f6c5, (0xdf1c bytes) 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4044 punch from 0x501076e to 0x50148f4, (0x4186 bytes) 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4045 punch from 0x5eb5941 to 0x5ebdc54, (0x8313 bytes) 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4046 punch from 0x864dc54 to 0x865c5b6, (0xe962 bytes) 2024-09-12T09:31:19.071 INFO:teuthology.orchestra.run.smithi007.stdout:4047 write 0xbbaf226 thru 0xbbb6440 (0x721b bytes) 2024-09-12T09:31:19.077 INFO:teuthology.orchestra.run.smithi007.stdout:4048 trunc from 0xc61f929 to 0xc00e24f 2024-09-12T09:31:19.087 INFO:teuthology.orchestra.run.smithi007.stdout:4049 write 0x29365ee thru 0x294310f (0xcb22 bytes) 2024-09-12T09:31:19.097 INFO:teuthology.orchestra.run.smithi007.stdout:4050 trunc from 0xc00e24f to 0xb598d49 2024-09-12T09:31:19.111 INFO:teuthology.orchestra.run.smithi007.stdout:4052 read 0x736a7e8 thru 0x7371607 (0x6e20 bytes) 2024-09-12T09:31:19.111 INFO:teuthology.orchestra.run.smithi007.stdout:4053 write 0x64243e8 thru 0x643354d (0xf166 bytes) 2024-09-12T09:31:19.120 INFO:teuthology.orchestra.run.smithi007.stdout:4055 trunc from 0xb598d49 to 0xeb8ca23 2024-09-12T09:31:19.130 INFO:teuthology.orchestra.run.smithi007.stdout:4056 punch from 0x2185bad to 0x2187381, (0x17d4 bytes) 2024-09-12T09:31:19.130 INFO:teuthology.orchestra.run.smithi007.stdout:4057 writesame 0x79b5300 thru 0x79be370 data_size 0x1(0x9071 bytes) 2024-09-12T09:31:19.154 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:19.150+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:4058 punch from 0xbace505 to 0xbad16fc, (0x31f7 bytes) 2024-09-12T09:31:19.181 INFO:teuthology.orchestra.run.smithi007.stdout:4059 writesame 0xadb24f thru 0xadd795 data_size 0x1(0x2547 bytes) 2024-09-12T09:31:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.183 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:4062 read 0xba37cc2 thru 0xba38497 (0x7d6 bytes) 2024-09-12T09:31:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:4064 trunc from 0xeb8ca23 to 0xe62213f 2024-09-12T09:31:19.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:19.216 INFO:teuthology.orchestra.run.smithi007.stdout:4065 trunc from 0xe62213f to 0x73635ad 2024-09-12T09:31:19.244 INFO:teuthology.orchestra.run.smithi007.stdout:4066 punch from 0x1d1ca37 to 0x1d24989, (0x7f52 bytes) 2024-09-12T09:31:19.252 INFO:teuthology.orchestra.run.smithi007.stdout:4067 trunc from 0x73635ad to 0x2ca96b6 2024-09-12T09:31:19.254 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7] out_osds: [6, 1, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:19.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:19.254 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-12T09:31:19.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-12T09:31:19.255 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:19.252+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:19.276 INFO:teuthology.orchestra.run.smithi007.stdout:4068 punch from 0x2a7e6aa to 0x2a85464, (0x6dba bytes) 2024-09-12T09:31:19.287 INFO:teuthology.orchestra.run.smithi007.stdout:4069 trunc from 0x2ca96b6 to 0x9a73670 2024-09-12T09:31:19.296 INFO:teuthology.orchestra.run.smithi007.stdout:4070 punch from 0x84ab98b to 0x84b3385, (0x79fa bytes) 2024-09-12T09:31:19.298 INFO:teuthology.orchestra.run.smithi007.stdout:4071 trunc from 0x9a73670 to 0xb07171f 2024-09-12T09:31:19.315 INFO:teuthology.orchestra.run.smithi007.stdout:4072 trunc from 0xb07171f to 0x60780c6 2024-09-12T09:31:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:19.325 INFO:teuthology.orchestra.run.smithi007.stdout:4074 writesame 0x5871513 thru 0x5873e2b data_size 0x15(0x2919 bytes) 2024-09-12T09:31:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:4077 punch from 0x1852a1b to 0x185309d, (0x682 bytes) 2024-09-12T09:31:19.332 INFO:teuthology.orchestra.run.smithi007.stdout:4079 punch from 0x5f91142 to 0x5f94226, (0x30e4 bytes) 2024-09-12T09:31:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.351 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:4080 write 0xc49e26a thru 0xc4a2ed0 (0x4c67 bytes) 2024-09-12T09:31:19.356 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:19.352+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:19.362 INFO:teuthology.orchestra.run.smithi007.stdout:4081 clone 39 order 25 su 1048576 sc 3 2024-09-12T09:31:19.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:19.457 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:19.454+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:19.532 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.532 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:19.580 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:31:19.580+0000 7f3454e55640 -1 mon.a@0(leader).osd e110 definitely_dead 0 2024-09-12T09:31:19.581 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:19.580+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:19.658 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:19.656+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:19.667 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.667 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:19.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:19.759 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:19.758+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:19.849 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:19.849 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:19.860 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:19.858+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:19.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:19.868 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:19.886 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:19.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:19.904 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:19.904 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:19.921 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:19.921 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:19.942 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:19.961 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:19.958+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:19.964 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:19.982 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:19.982 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:19.982 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:19.985 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:19.986 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:19.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:20.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:20.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:20.076 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:20.076+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:20.136 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:20.136 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:20.136 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:20.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:20.162 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:20.158+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.6. 2024-09-12T09:31:20.262 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:20.260+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:20.286 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-12T09:31:20.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:31:20.328 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.028682369999999, 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:20.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:20.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.846263452000002, 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:20.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.453476938999998, 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:20.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.105947051000001, 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:20.343 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:20.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.103493141000001, 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:20.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.101383253999998, 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:20.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:20.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.605636609000001, 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.575228323000001, 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:20.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:20.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:20.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.354263763999999, 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:20.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:20.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.655152481000002, 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:20.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.655135552000001, 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:20.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:20.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.616373954, 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:20.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:20.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.559005464999998, 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:20.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:20.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.558982046000001, 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:20.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:20.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:20.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.527753041, 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:20.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:20.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:20.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.446650084000002, 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:20.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.446624089, 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:20.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.193407601000001, 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:20.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:20.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.970723525, 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:20.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:20.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:20.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:20.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.970696716999999, 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.442 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:20.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:20.443 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:20.384+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:20.488 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:20.486+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:20.567 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:20.567 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:20.567 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:20.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:20.589 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:20.586+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:20.690 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:20.686+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:20.708 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:20.708 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:20.709 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:20.709 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:20.709 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:20.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:20.791 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:20.788+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:20.847 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148930 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 937984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.410677019000001, 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097427275999999993, 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148930 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435664+0000", 2024-09-12T09:31:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435666+0000", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435672+0000", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4969999999999998e-06 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435677+0000", 2024-09-12T09:31:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1780000000000002e-06 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532783+0000", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097105891 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532861+0000", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7760000000000001e-05 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533090+0000", 2024-09-12T09:31:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022925500000000001 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.410655843000001, 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.410630611999999, 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.410592692000002, 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.410562690999999, 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.402528270000001, 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.402207397000002, 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401783570999999, 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:20.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401732092, 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:20.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:20.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:20.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401703344000001, 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:20.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401152878000001, 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401110056, 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:20.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:20.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.401064165000001, 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:20.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.400552607000002, 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:20.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.400504827999999, 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:20.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.400449181999999, 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.400417161, 2024-09-12T09:31:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.399853012000001, 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.399459110999999, 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.399067413000001, 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:20.939 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:20.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:20.941 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:20.940+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:21.042 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:21.040+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.072 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:21.072 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:21.072 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:21.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:21.143 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:21.140+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:21.213 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.214 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:21.214 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:21.214 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:21.214 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:21.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:31:21.272 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:21.270+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:21.344 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:21.344+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:21.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.067509368000003, 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:21.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.608013138999993, 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.234596187999998, 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.234548941, 2024-09-12T09:31:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.234255864999994, 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.233693932999998, 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.233642570000001, 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.233069689999994, 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.232917391000001, 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.232560301000007, 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.231935331000003, 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.231883995000004, 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.231089205000004, 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.231014314999996, 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.230961995000001, 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.230913396999995, 2024-09-12T09:31:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.230658931999997, 2024-09-12T09:31:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.227204642000004, 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.169636095000001, 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 48.698364857999998, 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.463 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:21.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:21.465 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:21.462+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:21.565 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:21.564+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:21.636 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.636 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:21.637 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:21.637 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:21.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:21.696 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:21.694+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:21.767 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:21.762+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:21.767 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:21.768 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:21.768 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:21.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:21.868 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:21.866+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.540020053000006, 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:21.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.539924791000004, 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:21.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.536240167000003, 2024-09-12T09:31:21.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:21.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:21.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.534435076999998, 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:21.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:21.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.533888402000002, 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:21.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:21.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.533119959000004, 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.532709591, 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.532570414999995, 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:21.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.524791604000001, 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:21.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:21.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:21.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.522427590000007, 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:21.999 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:22.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:22.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.511675697000001, 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.511479305999998, 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:22.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:22.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.509984885999998, 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:22.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.809742025000006, 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:22.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:22.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:22.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.809719607000005, 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:22.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.809696466000005, 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:22.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:22.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:22.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.809671507000004, 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:22.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.037 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.766319993000003, 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:22.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:22.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:22.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.149841950999999, 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.044 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:22.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.149742660999998, 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:22.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:22.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:22.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:22.055 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:22.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:22.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:22.057 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:22.056+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:22.157 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:22.154+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:22.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:22.217 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:22.228 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:22.228 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:22.228 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:22.234 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:22.234 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:22.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:22.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:22.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:22.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:22.284 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:22.282+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:22.293 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:22.294 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:22.312 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:22.312 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:22.329 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:22.330 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:22.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:22.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:22.359 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:22.356+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:22.414 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone38 from 0xc4a2ed1 (overlap 0x701b6) to 0x64e57 2024-09-12T09:31:22.423 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:22.424 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:22.442 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:22.460 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:22.459+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:22.560 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:22.558+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:22.580 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:22.580 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:22.587 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-12T09:31:22.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:22.599 INFO:teuthology.orchestra.run.smithi007.stdout:4082 write 0x1ef933f thru 0x1f06372 (0xd034 bytes) 2024-09-12T09:31:22.625 INFO:teuthology.orchestra.run.smithi007.stdout:4083 clone 40 order 24 su 262144 sc 7 2024-09-12T09:31:22.662 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:22.659+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:22.763 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:22.758+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:22.782 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:22.782 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:22.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:22.864 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:22.863+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:22.965 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:22.963+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:22.970 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:22.970 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:22.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:23.066 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:23.063+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:23.120 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.120 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:23.184 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:23.183+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:23.269 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:23.267+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:23.277 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.277 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:31:23.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:23.369 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:23.366+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:23.470 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.470 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.470 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:23.466+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:23.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:23.571 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:23.567+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:23.673 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:23.670+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:23.773 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:23.770+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:23.798 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.798 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:23.874 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:23.871+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:23.936 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:23.936 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:23.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:23.992 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:23.991+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:24.077 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:24.074+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:24.113 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:24.113 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:24.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:24.177 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:24.175+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:24.249 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:24.249 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:24.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:24.308 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:24.307+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:24.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:24.378 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:24.374+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:24.437 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:24.437 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:24.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:24.479 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:24.474+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:24.518 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:24.537 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:24.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:24.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:24.579 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:24.575+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:51759 3.0 e53/23 3:02bd63b0:::rbd_data.1191a32cdd0c.0000000000000013:head v 53'63, mlcod=49'61)", 2024-09-12T09:31:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.284207667000004, 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014204095999999999, 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.299414+0000", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54e-07 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303696+0000", 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042816140000000004 2024-09-12T09:31:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303717+0000", 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1480000000000001e-05 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303728+0000", 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1209999999999999e-05 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303804+0000", 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5761999999999997e-05 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.303833+0000", 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8852000000000001e-05 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313565+0000", 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097315430000000005 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:08.313618+0000", 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3480999999999998e-05 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:54569 3.9 e55/23 3:9625be71:::rbd_data.1191175d8602.000000000000000b:head v 55'54, mlcod=49'52)", 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.101788748999994, 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014438558000000001, 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481833+0000", 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.481834+0000", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3909999999999999e-06 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489339+0000", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075044279999999996 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489369+0000", 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9805999999999999e-05 2024-09-12T09:31:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489383+0000", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4712000000000001e-05 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489478+0000", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4961000000000001e-05 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.489498+0000", 2024-09-12T09:31:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9596999999999999e-05 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496131+0000", 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066327469999999996 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:10.496271+0000", 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014091599999999999 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.709002236000003, 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:31:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.361472347999999, 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:31:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:31:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.359018438, 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:31:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.356908550999997, 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:31:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.861161906, 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:31:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:31:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4497.0:141950 3.4 e88/83 ondisk, result = 0)", 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.830753620000003, 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020160239, 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752868+0000", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752871+0000", 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752882+0000", 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1562999999999999e-05 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752921+0000", 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9338999999999997e-05 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.752943+0000", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1596000000000001e-05 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772813+0000", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019870364000000001 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772832+0000", 2024-09-12T09:31:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8660000000000001e-05 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773028+0000", 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000196207 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:142537 3.0 3:0024293e:::rbd_data.1191f803704b.000000000000019d:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.609789061000001, 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764371, 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142537 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973833+0000", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973835+0000", 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.553e-06 2024-09-12T09:31:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973848+0000", 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973883+0000", 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4632999999999999e-05 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.973899+0000", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5763000000000001e-05 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974030+0000", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013107900000000001 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.974249+0000", 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021872100000000001 2024-09-12T09:31:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.977616+0000", 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033673520000000001 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990393+0000", 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012777337999999999 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990424+0000", 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0667000000000003e-05 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990520+0000", 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6112000000000004e-05 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:52.990597+0000", 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6913e-05 2024-09-12T09:31:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149795 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.910677778, 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013904685999999999, 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149795 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672944+0000", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672947+0000", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8200000000000001e-06 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672952+0000", 2024-09-12T09:31:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672956+0000", 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3670000000000001e-06 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678845+0000", 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058889759999999998 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.678893+0000", 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8352000000000001e-05 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679166+0000", 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027324700000000001 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680207+0000", 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010408870000000001 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686738+0000", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065303840000000002 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686807+0000", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9181000000000001e-05 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686849+0000", 2024-09-12T09:31:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1832999999999997e-05 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149796 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.910660848999999, 2024-09-12T09:31:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014003075, 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149796 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672961+0000", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672963+0000", 2024-09-12T09:31:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8279999999999999e-06 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672966+0000", 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2179999999999999e-06 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.672975+0000", 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0219999999999993e-06 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679508+0000", 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065324959999999996 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679584+0000", 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6332e-05 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.679848+0000", 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026406500000000001 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.680456+0000", 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060777400000000005 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686891+0000", 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064353029999999999 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686932+0000", 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.686964+0000", 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1742999999999997e-05 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149836 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.871899250999999, 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013787506, 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149836 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711722+0000", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711723+0000", 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711726+0000", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.264e-06 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711734+0000", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5979999999999999e-06 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.711739+0000", 2024-09-12T09:31:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1749999999999996e-06 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718611+0000", 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068715879999999997 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.718698+0000", 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7522000000000007e-05 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719121+0000", 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042273199999999999 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.719793+0000", 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00067174699999999999 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725454+0000", 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056613339999999996 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725486+0000", 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2007e-05 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.725510+0000", 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4051000000000001e-05 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149910 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.814530762, 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013748081000000001, 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149910 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:24.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769091+0000", 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769094+0000", 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769100+0000", 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769106+0000", 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776154+0000", 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070472920000000001 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776200+0000", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6183000000000002e-05 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776396+0000", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019668499999999999 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777892+0000", 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014960069999999999 2024-09-12T09:31:24.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782740+0000", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048477930000000004 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782798+0000", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7728e-05 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782839+0000", 2024-09-12T09:31:24.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1010999999999997e-05 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149911 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.814507342999999, 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013907443, 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149911 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769114+0000", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769115+0000", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769118+0000", 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7989999999999998e-06 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769124+0000", 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4520000000000002e-06 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.769129+0000", 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.776942+0000", 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007813726 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777043+0000", 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010068000000000001 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.777426+0000", 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038273499999999999 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.778110+0000", 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068451199999999997 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782902+0000", 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047919590000000001 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.782978+0000", 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5328999999999997e-05 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.783022+0000", 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4230000000000002e-05 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149951 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:24.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.783278337999999, 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015432023, 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149951 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800343+0000", 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800344+0000", 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800346+0000", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7080000000000002e-06 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800354+0000", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0139999999999998e-06 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.800359+0000", 2024-09-12T09:31:24.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.831e-06 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808521+0000", 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081621880000000008 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808582+0000", 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0322000000000003e-05 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.808789+0000", 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020759700000000001 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.809413+0000", 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062395400000000002 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815726+0000", 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063125619999999999 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815755+0000", 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9011000000000001e-05 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.815775+0000", 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0448000000000001e-05 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150051 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.702175381, 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014307736999999999, 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150051 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881446+0000", 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881447+0000", 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881449+0000", 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7049999999999999e-06 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881457+0000", 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4699999999999996e-06 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881462+0000", 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.57e-06 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889637+0000", 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081749040000000002 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.889725+0000", 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7593e-05 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890090+0000", 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000364836 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891777+0000", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016874979999999999 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895662+0000", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038848749999999999 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895714+0000", 2024-09-12T09:31:24.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619000000000002e-05 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895754+0000", 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0290000000000002e-05 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150052 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.702149385999999, 2024-09-12T09:31:24.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014434728000000001, 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150052 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:24.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881472+0000", 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881473+0000", 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-07 2024-09-12T09:31:24.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881476+0000", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0850000000000001e-06 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881482+0000", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6529999999999997e-06 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.881488+0000", 2024-09-12T09:31:24.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890603+0000", 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091149070000000002 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.890682+0000", 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9288000000000004e-05 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891097+0000", 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041502999999999999 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.891902+0000", 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080524700000000004 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895807+0000", 2024-09-12T09:31:24.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003904804 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895867+0000", 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0127000000000002e-05 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.895907+0000", 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9956999999999997e-05 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150458 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.448932897999999, 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016029277000000001, 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150458 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134689+0000", 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134691+0000", 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6360000000000001e-06 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134693+0000", 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8629999999999999e-06 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.134696+0000", 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3309999999999998e-06 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144366+0000", 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096697369999999994 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144453+0000", 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7472000000000005e-05 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.144848+0000", 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039482800000000001 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.145623+0000", 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077489400000000004 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150630+0000", 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005006992 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150677+0000", 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7460000000000003e-05 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.150718+0000", 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.083e-05 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150680 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.226248821999999, 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014152461999999999, 2024-09-12T09:31:24.697 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150680 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:24.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357373+0000", 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0800000000000005e-07 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357377+0000", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.309e-06 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357384+0000", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4490000000000002e-06 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357391+0000", 2024-09-12T09:31:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8929999999999999e-06 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364205+0000", 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068144549999999996 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364299+0000", 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3381999999999995e-05 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.364746+0000", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00044694199999999998 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.368706+0000", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039600479999999999 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371454+0000", 2024-09-12T09:31:24.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002748402 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371487+0000", 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2793999999999998e-05 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371525+0000", 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8279999999999999e-05 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:150681 3.4 3:247e8aad:::rbd_object_map.119134d20ad9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.226222014000001, 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014315738999999999, 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 150681 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357400+0000", 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357403+0000", 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6979999999999998e-06 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357410+0000", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.357415+0000", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365399+0000", 2024-09-12T09:31:24.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079846280000000006 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365474+0000", 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4345999999999996e-05 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.365696+0000", 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022181999999999999 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.369057+0000", 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003361232 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371596+0000", 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025392230000000002 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:24.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371628+0000", 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1900999999999998e-05 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:01.371715+0000", 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7498999999999997e-05 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.710 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:24.711 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:24.712 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:24.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:24.713 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:24.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:24.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:31:24.716 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone39 from 0xc4a2ed1 (overlap 0xc4a2ed1) to 0x0 2024-09-12T09:31:24.716 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:24.711+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:24.813 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:24.811+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3.2498125e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.072006120000000007, 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.072051339000000006, 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:24.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:24.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:31:24.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:24.892 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:24.892 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-12T09:31:24.892 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:24.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:24.948 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:24.947+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:25.015 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:25.014+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:25.073 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:25.073 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:25.073 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:25.073 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:25.073 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:25.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:25.115 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:25.114+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:25.216 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:25.214+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148930 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 937984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.780901817, 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097427275999999993, 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148930 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435663+0000", 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435664+0000", 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435666+0000", 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435672+0000", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4969999999999998e-06 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435677+0000", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1780000000000002e-06 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532783+0000", 2024-09-12T09:31:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097105891 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.532861+0000", 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7760000000000001e-05 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533090+0000", 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022925500000000001 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.780880641, 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:25.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.780855410000001, 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:25.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:25.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.78081749, 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:25.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:25.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:25.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.780787489000002, 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:25.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:25.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.772753068, 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:25.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.772432195, 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:25.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:25.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.772008369000002, 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:25.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:25.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.771956889999998, 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:25.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.771928142, 2024-09-12T09:31:25.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:25.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:25.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:25.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.771377676, 2024-09-12T09:31:25.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:25.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:25.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.272 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.771334853999999, 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:25.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.771288963, 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:25.277 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:25.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.770777405, 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:25.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:25.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.770729626000001, 2024-09-12T09:31:25.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:25.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.770673980000002, 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:25.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:25.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.770641958999999, 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:25.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.77007781, 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:25.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.769683909000001, 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:25.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.769292211, 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:25.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:25.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:25.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:25.318 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:25.318 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6] out_osds: [1, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:25.318 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:25.318 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-12T09:31:25.318 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-12T09:31:25.318 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-12T09:31:25.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-12T09:31:25.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:25.321 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:25.319+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:25.421 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:25.419+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:25.451 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:25.451 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:25.451 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:25.451 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:25.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:25.494 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-12T09:31:25.494 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:25.552 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:25.551+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:25.582 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:25.582 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:25.582 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:25.582 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:25.582 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:25.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:25.640 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:25.638+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.430163586999996, 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.970667358, 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.597250407000004, 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.597203160000007, 2024-09-12T09:31:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.596910084000001, 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.596348152000004, 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.596296788999993, 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.595723909, 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.595571609999993, 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.595214519999999, 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.594589549999995, 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.594538213999996, 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:25.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:25.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.593743423999996, 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:25.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:25.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.593668534000003, 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:25.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:25.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:25.789 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.593616213999994, 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:25.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:25.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.593567616000001, 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:25.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:25.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:25.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.593313151000004, 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:25.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:25.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.589858860999996, 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:25.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.532290314000001, 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:25.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:25.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:25.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 53.061019076999997, 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:25.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:25.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:25.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:25.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:25.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:25.822 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:25.823 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-12T09:31:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:4085 punch from 0xc1f40fa to 0xc1fe0f7, (0x9ffd bytes) 2024-09-12T09:31:25.824 INFO:teuthology.orchestra.run.smithi007.stdout:4088 write 0xea1331 thru 0xeb131d (0xffed bytes) 2024-09-12T09:31:25.824 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:25.823+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:25.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:25.840 INFO:teuthology.orchestra.run.smithi007.stdout:4089 write 0x3f611ca thru 0x3f70578 (0xf3af bytes) 2024-09-12T09:31:25.877 INFO:teuthology.orchestra.run.smithi007.stdout:4090 write 0xb0b94fd thru 0xb0bb99f (0x24a3 bytes) 2024-09-12T09:31:25.888 INFO:teuthology.orchestra.run.smithi007.stdout:4092 punch from 0x5d03ad2 to 0x5d0a047, (0x6575 bytes) 2024-09-12T09:31:25.888 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:25.889 INFO:teuthology.orchestra.run.smithi007.stdout:4095 write 0xbe3b1af thru 0xbe41b11 (0x6963 bytes) 2024-09-12T09:31:25.903 INFO:teuthology.orchestra.run.smithi007.stdout:4098 read 0x6b50505 thru 0x6b6004e (0xfb4a bytes) 2024-09-12T09:31:25.924 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:25.922+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:25.930 INFO:teuthology.orchestra.run.smithi007.stdout:4099 trunc from 0xc4a2ed1 to 0x10df48a 2024-09-12T09:31:26.008 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:26.008 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:26.020 INFO:teuthology.orchestra.run.smithi007.stdout:4100 punch from 0xaeb2e8 to 0xaf9398, (0xe0b0 bytes) 2024-09-12T09:31:26.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:26.045 INFO:teuthology.orchestra.run.smithi007.stdout:4101 trunc from 0x10df48a to 0xa345c59 2024-09-12T09:31:26.068 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:26.066+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:26.078 INFO:teuthology.orchestra.run.smithi007.stdout:4103 trunc from 0xa345c59 to 0x20ea8c4 2024-09-12T09:31:26.121 INFO:teuthology.orchestra.run.smithi007.stdout:4104 trunc from 0x20ea8c4 to 0xc861310 2024-09-12T09:31:26.126 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:26.123+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:26.136 INFO:teuthology.orchestra.run.smithi007.stdout:4105 write 0x14083a5 thru 0x1409505 (0x1161 bytes) 2024-09-12T09:31:26.138 INFO:teuthology.orchestra.run.smithi007.stdout:4107 read 0x89cac33 thru 0x89d9e77 (0xf245 bytes) 2024-09-12T09:31:26.138 INFO:teuthology.orchestra.run.smithi007.stdout:4108 punch from 0x733dd04 to 0x7344f40, (0x723c bytes) 2024-09-12T09:31:26.138 INFO:teuthology.orchestra.run.smithi007.stdout:4109 read 0x8960da8 thru 0x8967930 (0x6b89 bytes) 2024-09-12T09:31:26.169 INFO:teuthology.orchestra.run.smithi007.stdout:4110 trunc from 0xc861310 to 0x74b2ab 2024-09-12T09:31:26.190 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:26.191 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:26.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:26.224 INFO:teuthology.orchestra.run.smithi007.stdout:4111 write 0x9d4e330 thru 0x9d4e9dc (0x6ad bytes) 2024-09-12T09:31:26.230 INFO:teuthology.orchestra.run.smithi007.stdout:4114 punch from 0x73de772 to 0x73e93fd, (0xac8b bytes) 2024-09-12T09:31:26.234 INFO:teuthology.orchestra.run.smithi007.stdout:4115 trunc from 0x9d4e9dd to 0xb53bab2 2024-09-12T09:31:26.246 INFO:teuthology.orchestra.run.smithi007.stdout:4116 read 0x6694ac0 thru 0x66a0de7 (0xc328 bytes) 2024-09-12T09:31:26.247 INFO:teuthology.orchestra.run.smithi007.stdout:4117 read 0x2dc9526 thru 0x2dd3ac2 (0xa59d bytes) 2024-09-12T09:31:26.248 INFO:teuthology.orchestra.run.smithi007.stdout:4118 punch from 0x9146b54 to 0x9146fd4, (0x480 bytes) 2024-09-12T09:31:26.248 INFO:teuthology.orchestra.run.smithi007.stdout:4119 write 0x9f42aa thru 0x9f7cf9 (0x3a50 bytes) 2024-09-12T09:31:26.248 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:26.246+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:26.251 INFO:teuthology.orchestra.run.smithi007.stdout:4120 read 0x8d6cc0c thru 0x8d7a8b8 (0xdcad bytes) 2024-09-12T09:31:26.251 INFO:teuthology.orchestra.run.smithi007.stdout:4123 write 0x3d7690e thru 0x3d80f93 (0xa686 bytes) 2024-09-12T09:31:26.254 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.273 INFO:teuthology.orchestra.run.smithi007.stdout:4126 writesame 0xdef6ad0 thru 0xdef813d data_size 0x3a(0x166e bytes) 2024-09-12T09:31:26.273 INFO:teuthology.orchestra.run.smithi007.stdout:4127 read 0xb2ab788 thru 0xb2b31f1 (0x7a6a bytes) 2024-09-12T09:31:26.273 INFO:teuthology.orchestra.run.smithi007.stdout:4129 write 0x30f4c7b thru 0x30ff895 (0xac1b bytes) 2024-09-12T09:31:26.277 INFO:teuthology.orchestra.run.smithi007.stdout:4130 write 0xd87d0af thru 0xd87d632 (0x584 bytes) 2024-09-12T09:31:26.283 INFO:teuthology.orchestra.run.smithi007.stdout:4133 read 0xa499765 thru 0xa4a5d0e (0xc5aa bytes) 2024-09-12T09:31:26.283 INFO:teuthology.orchestra.run.smithi007.stdout:4136 write 0x706c109 thru 0x706faac (0x39a4 bytes) 2024-09-12T09:31:26.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.289 INFO:teuthology.orchestra.run.smithi007.stdout:4139 punch from 0x2c48678 to 0x2c53a7d, (0xb405 bytes) 2024-09-12T09:31:26.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.289 INFO:teuthology.orchestra.run.smithi007.stdout:4142 punch from 0xa02cb82 to 0xa02ef26, (0x23a4 bytes) 2024-09-12T09:31:26.289 INFO:teuthology.orchestra.run.smithi007.stdout:4143 writesame 0x2c201c2 thru 0x2c2264d data_size 0x2(0x248c bytes) 2024-09-12T09:31:26.304 INFO:teuthology.orchestra.run.smithi007.stdout:4144 trunc from 0xdef813e to 0xa6a8d54 2024-09-12T09:31:26.328 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:26.327+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:26.330 INFO:teuthology.orchestra.run.smithi007.stdout:4145 writesame 0xa70858c thru 0xa70fe4b data_size 0x60(0x78c0 bytes) 2024-09-12T09:31:26.330 INFO:teuthology.orchestra.run.smithi007.stdout:4148 writesame 0x43d5b9a thru 0x43de6aa data_size 0x1(0x8b11 bytes) 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.924439277000005, 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:26.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:26.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:26.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:26.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.924344015000003, 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:26.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:26.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.920659391000001, 2024-09-12T09:31:26.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:26.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.918854300999996, 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.918307626000001, 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.917539183000002, 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.917128814999998, 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.916989638999993, 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.909210827999999, 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.906846814000005, 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.896094921, 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.895898529999997, 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 96.894404109999996, 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.194161249000004, 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.194138831000004, 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.194115690000004, 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.194090731000003, 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 63.150739217000002, 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.534261174999997, 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.534161885000003, 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.444 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:26.445 INFO:teuthology.orchestra.run.smithi007.stdout:4149 writesame 0xaaf6689 thru 0xab005d9 data_size 0x5(0x9f51 bytes) 2024-09-12T09:31:26.445 INFO:teuthology.orchestra.run.smithi007.stdout:4150 write 0xd0bbe7c thru 0xd0bf6d3 (0x3858 bytes) 2024-09-12T09:31:26.445 INFO:teuthology.orchestra.run.smithi007.stdout:4151 trunc from 0xd0bf6d4 to 0xc03ef4 2024-09-12T09:31:26.446 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:26.446 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:26.443+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:26.462 INFO:teuthology.orchestra.run.smithi007.stdout:4152 read 0xae3d53 thru 0xae3d68 (0x16 bytes) 2024-09-12T09:31:26.462 INFO:teuthology.orchestra.run.smithi007.stdout:4153 read 0x941766 thru 0x94852e (0x6dc9 bytes) 2024-09-12T09:31:26.487 INFO:teuthology.orchestra.run.smithi007.stdout:4154 write 0x3fbbc1a thru 0x3fbe807 (0x2bee bytes) 2024-09-12T09:31:26.491 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.491 INFO:teuthology.orchestra.run.smithi007.stdout:4156 punch from 0x34a7a2f to 0x34acf76, (0x5547 bytes) 2024-09-12T09:31:26.491 INFO:teuthology.orchestra.run.smithi007.stdout:4160 write 0x33cac1 thru 0x346583 (0x9ac3 bytes) 2024-09-12T09:31:26.527 INFO:teuthology.orchestra.run.smithi007.stdout:4162 write 0xa66a907 thru 0xa673fba (0x96b4 bytes) 2024-09-12T09:31:26.533 INFO:teuthology.orchestra.run.smithi007.stdout:4164 punch from 0x6b24ac9 to 0x6b2f510, (0xaa47 bytes) 2024-09-12T09:31:26.533 INFO:teuthology.orchestra.run.smithi007.stdout:4165 read 0x43e6c13 thru 0x43f4673 (0xda61 bytes) 2024-09-12T09:31:26.534 INFO:teuthology.orchestra.run.smithi007.stdout:4166 writesame 0x6497ab6 thru 0x6498994 data_size 0x1b(0xedf bytes) 2024-09-12T09:31:26.537 INFO:teuthology.orchestra.run.smithi007.stdout:4167 read 0x37f23aa thru 0x37f9975 (0x75cc bytes) 2024-09-12T09:31:26.538 INFO:teuthology.orchestra.run.smithi007.stdout:4169 read 0x4fb0135 thru 0x4fbe8e9 (0xe7b5 bytes) 2024-09-12T09:31:26.539 INFO:teuthology.orchestra.run.smithi007.stdout:4171 read 0x5affc79 thru 0x5b05093 (0x541b bytes) 2024-09-12T09:31:26.546 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:26.543+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:4172 write 0xe23dce4 thru 0xe24564e (0x796b bytes) 2024-09-12T09:31:26.577 INFO:teuthology.orchestra.run.smithi007.stdout:4174 trunc from 0xe24564f to 0xc6701d5 2024-09-12T09:31:26.607 INFO:teuthology.orchestra.run.smithi007.stdout:4175 write 0x6f52ac8 thru 0x6f57a3c (0x4f75 bytes) 2024-09-12T09:31:26.626 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.626 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:26.626 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:26.626 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:26.631 INFO:teuthology.orchestra.run.smithi007.stdout:4177 trunc from 0xc6701d5 to 0x58e02ec 2024-09-12T09:31:26.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:26.656 INFO:teuthology.orchestra.run.smithi007.stdout:4178 read 0x4f47fca thru 0x4f5692d (0xe964 bytes) 2024-09-12T09:31:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.657 INFO:teuthology.orchestra.run.smithi007.stdout:4183 writesame 0x38c6a75 thru 0x38cc0bd data_size 0x25(0x5649 bytes) 2024-09-12T09:31:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:4184 write 0xf05e5f thru 0xf10250 (0xa3f2 bytes) 2024-09-12T09:31:26.664 INFO:teuthology.orchestra.run.smithi007.stdout:4185 punch from 0x17f8561 to 0x17ff192, (0x6c31 bytes) 2024-09-12T09:31:26.664 INFO:teuthology.orchestra.run.smithi007.stdout:4187 read 0x33fb8fd thru 0x33fd675 (0x1d79 bytes) 2024-09-12T09:31:26.664 INFO:teuthology.orchestra.run.smithi007.stdout:4189 punch from 0x24440c7 to 0x244ae85, (0x6dbe bytes) 2024-09-12T09:31:26.665 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.665 INFO:teuthology.orchestra.run.smithi007.stdout:4193 read 0x2c90b74 thru 0x2c96c7f (0x610c bytes) 2024-09-12T09:31:26.665 INFO:teuthology.orchestra.run.smithi007.stdout:4194 read 0x931082 thru 0x9394ec (0x846b bytes) 2024-09-12T09:31:26.666 INFO:teuthology.orchestra.run.smithi007.stdout:4196 read 0x36f64c4 thru 0x37029e2 (0xc51f bytes) 2024-09-12T09:31:26.684 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:26.682+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:26.693 INFO:teuthology.orchestra.run.smithi007.stdout:4197 writesame 0xb0fc34b thru 0xb0fe538 data_size 0x2(0x21ee bytes) 2024-09-12T09:31:26.693 INFO:teuthology.orchestra.run.smithi007.stdout:4201 punch from 0x8f48758 to 0x8f56e2e, (0xe6d6 bytes) 2024-09-12T09:31:26.693 INFO:teuthology.orchestra.run.smithi007.stdout:4202 write 0x9cdcd7a thru 0x9ce4c4a (0x7ed1 bytes) 2024-09-12T09:31:26.699 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:26.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:31:26.728 INFO:teuthology.orchestra.run.smithi007.stdout:4205 trunc from 0xb0fe539 to 0xc9dc9a 2024-09-12T09:31:26.747 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:26.742+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:26.757 INFO:teuthology.orchestra.run.smithi007.stdout:4206 trunc from 0xc9dc9a to 0x25967d2 2024-09-12T09:31:26.772 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:26.773 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:26.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:26.788 INFO:teuthology.orchestra.run.smithi007.stdout:4207 write 0x6812a2f thru 0x6818b22 (0x60f4 bytes) 2024-09-12T09:31:26.790 INFO:teuthology.orchestra.run.smithi007.stdout:4208 read 0x3d20deb thru 0x3d25dce (0x4fe4 bytes) 2024-09-12T09:31:26.791 INFO:teuthology.orchestra.run.smithi007.stdout:4209 punch from 0x3f81a0f to 0x3f857c7, (0x3db8 bytes) 2024-09-12T09:31:26.791 INFO:teuthology.orchestra.run.smithi007.stdout:4211 writesame 0x8c33 thru 0x181a7 data_size 0x1(0xf575 bytes) 2024-09-12T09:31:26.832 INFO:teuthology.orchestra.run.smithi007.stdout:4213 read 0x4e7ea36 thru 0x4e83484 (0x4a4f bytes) 2024-09-12T09:31:26.832 INFO:teuthology.orchestra.run.smithi007.stdout:4214 read 0x1831ab0 thru 0x183e53e (0xca8f bytes) 2024-09-12T09:31:26.838 INFO:teuthology.orchestra.run.smithi007.stdout:4216 trunc from 0x6818b23 to 0x8837981 2024-09-12T09:31:26.846 INFO:teuthology.orchestra.run.smithi007.stdout:4218 writesame 0x1d7c85b thru 0x1d80321 data_size 0x1(0x3ac7 bytes) 2024-09-12T09:31:26.859 INFO:teuthology.orchestra.run.smithi007.stdout:4219 write 0x209e203 thru 0x20a59b6 (0x77b4 bytes) 2024-09-12T09:31:26.861 INFO:teuthology.orchestra.run.smithi007.stdout:4221 write 0x816bcb3 thru 0x817587e (0x9bcc bytes) 2024-09-12T09:31:26.866 INFO:teuthology.orchestra.run.smithi007.stdout:4222 punch from 0x414f261 to 0x4152eb5, (0x3c54 bytes) 2024-09-12T09:31:26.866 INFO:teuthology.orchestra.run.smithi007.stdout:4223 punch from 0x72f0b3 to 0x73399f, (0x48ec bytes) 2024-09-12T09:31:26.866 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:26.866 INFO:teuthology.orchestra.run.smithi007.stdout:4225 write 0x30ad4d thru 0x3190f3 (0xe3a7 bytes) 2024-09-12T09:31:26.876 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:26.874+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:26.877 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:26.879 INFO:teuthology.orchestra.run.smithi007.stdout:4226 writesame 0xb0e42c7 thru 0xb0f2339 data_size 0x6b(0xe073 bytes) 2024-09-12T09:31:26.880 INFO:teuthology.orchestra.run.smithi007.stdout:4228 read 0x21b3025 thru 0x21bbe70 (0x8e4c bytes) 2024-09-12T09:31:26.880 INFO:teuthology.orchestra.run.smithi007.stdout:4229 writesame 0x36c5e84 thru 0x36c92a7 data_size 0x47(0x3424 bytes) 2024-09-12T09:31:26.881 INFO:teuthology.orchestra.run.smithi007.stdout:4230 clone 41 order 18 su 8192 sc 10 2024-09-12T09:31:26.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:26.895 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:26.912 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:26.912 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:26.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:26.929 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:26.948 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:26.947+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:26.949 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:26.949 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:26.956 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:26.956 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:26.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316936, 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:26.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:26.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316886392, 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:26.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.31685265, 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:26.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:26.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.970 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316805612, 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:26.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:26.975 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316747772, 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:26.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316688685, 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.31665449800001, 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:26.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316623162, 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:26.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316593121, 2024-09-12T09:31:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:26.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:26.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:26.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.31656187599999, 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:26.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:27.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:27.002 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316520107, 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:27.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:27.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.31648864100001, 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:27.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316441101, 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:27.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:27.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:27.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316408064, 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:27.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:27.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316363044, 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:27.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:27.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:27.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316330386, 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:27.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316298571, 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:27.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316240658, 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:27.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:27.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316198676, 2024-09-12T09:31:27.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:27.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.316162694, 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:27.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:27.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:27.048 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:27.049 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:27.050 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:27.050 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:27.050 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:27.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:27.051 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:27.052 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:27.052 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:27.047+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:27.151 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:27.146+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:27.223 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:27.223 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:27.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:27.280 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:27.278+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:27.353 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:27.351+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:27.403 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:27.403 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:27.417 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:27.464 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:27.462+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:27.535 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:27.535 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:27.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:27.596 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:27.595+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:27.655 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:27.654+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:27.685 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:27.686 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:31:27.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:27.768 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:27.767+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:27.857 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:27.854+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:27.890 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:27.890 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:27.893 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone40 from 0xb0f233a (overlap 0x74b2ab) to 0x2eadde 2024-09-12T09:31:27.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:27.958 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:27.955+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:28.034 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.034 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:28.078 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-12T09:31:28.079 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:28.075+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:28.081 INFO:teuthology.orchestra.run.smithi007.stdout:4233 punch from 0x9992740 to 0x9994daf, (0x266f bytes) 2024-09-12T09:31:28.088 INFO:teuthology.orchestra.run.smithi007.stdout:4234 read 0xa6caca9 thru 0xa6d32e4 (0x863c bytes) 2024-09-12T09:31:28.088 INFO:teuthology.orchestra.run.smithi007.stdout:4235 punch from 0x6838855 to 0x68418ff, (0x90aa bytes) 2024-09-12T09:31:28.106 INFO:teuthology.orchestra.run.smithi007.stdout:4236 trunc from 0xb0f233a to 0x3281e50 2024-09-12T09:31:28.160 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:28.159+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:28.173 INFO:teuthology.orchestra.run.smithi007.stdout:4237 trunc from 0x3281e50 to 0x398e2bd 2024-09-12T09:31:28.183 INFO:teuthology.orchestra.run.smithi007.stdout:4239 punch from 0xf2a447 to 0xf2bfd4, (0x1b8d bytes) 2024-09-12T09:31:28.183 INFO:teuthology.orchestra.run.smithi007.stdout:4240 read 0x1c6d97d thru 0x1c7b3b0 (0xda34 bytes) 2024-09-12T09:31:28.200 INFO:teuthology.orchestra.run.smithi007.stdout:4241 trunc from 0x398e2bd to 0xc09c50b 2024-09-12T09:31:28.213 INFO:teuthology.orchestra.run.smithi007.stdout:4242 trunc from 0xc09c50b to 0xb490963 2024-09-12T09:31:28.218 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.218 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.226 INFO:teuthology.orchestra.run.smithi007.stdout:4243 read 0x25c8c21 thru 0x25ccea5 (0x4285 bytes) 2024-09-12T09:31:28.227 INFO:teuthology.orchestra.run.smithi007.stdout:4244 writesame 0x58ca269 thru 0x58cff3e data_size 0x6(0x5cd6 bytes) 2024-09-12T09:31:28.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:28.238 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:28.247 INFO:teuthology.orchestra.run.smithi007.stdout:4246 trunc from 0xb490963 to 0x547c710 2024-09-12T09:31:28.261 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:28.258+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:28.289 INFO:teuthology.orchestra.run.smithi007.stdout:4247 writesame 0xa0878c9 thru 0xa08967b data_size 0x1(0x1db3 bytes) 2024-09-12T09:31:28.289 INFO:teuthology.orchestra.run.smithi007.stdout:4248 writesame 0x24df4eb thru 0x24e2201 data_size 0x11(0x2d17 bytes) 2024-09-12T09:31:28.312 INFO:teuthology.orchestra.run.smithi007.stdout:4250 read 0x2abafdb thru 0x2abba2d (0xa53 bytes) 2024-09-12T09:31:28.312 INFO:teuthology.orchestra.run.smithi007.stdout:4251 write 0x71c222 thru 0x727379 (0xb158 bytes) 2024-09-12T09:31:28.347 INFO:teuthology.orchestra.run.smithi007.stdout:4253 write 0x8f9fe2b thru 0x8faab21 (0xacf7 bytes) 2024-09-12T09:31:28.362 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:28.358+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:28.364 INFO:teuthology.orchestra.run.smithi007.stdout:4255 write 0x220e3ee thru 0x2214a40 (0x6653 bytes) 2024-09-12T09:31:28.371 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.371 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:28.403 INFO:teuthology.orchestra.run.smithi007.stdout:4256 write 0xe59780a thru 0xe597841 (0x38 bytes) 2024-09-12T09:31:28.430 INFO:teuthology.orchestra.run.smithi007.stdout:4257 trunc from 0xe597842 to 0x13f3188 2024-09-12T09:31:28.462 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:28.459+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:28.502 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:28.523 INFO:teuthology.orchestra.run.smithi007.stdout:4260 trunc from 0x13f3188 to 0xc9b2683 2024-09-12T09:31:28.535 INFO:teuthology.orchestra.run.smithi007.stdout:4261 writesame 0x9cba096 thru 0x9cc3232 data_size 0x1(0x919d bytes) 2024-09-12T09:31:28.539 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.540 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:28.563 INFO:teuthology.orchestra.run.smithi007.stdout:4262 writesame 0x1d626ce thru 0x1d6635a data_size 0x1(0x3c8d bytes) 2024-09-12T09:31:28.563 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:28.558+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:28.582 INFO:teuthology.orchestra.run.smithi007.stdout:4263 write 0x152e47a thru 0x153abc8 (0xc74f bytes) 2024-09-12T09:31:28.599 INFO:teuthology.orchestra.run.smithi007.stdout:4264 writesame 0x4e57cf4 thru 0x4e5e9ef data_size 0x96(0x6cfc bytes) 2024-09-12T09:31:28.615 INFO:teuthology.orchestra.run.smithi007.stdout:4265 read 0xc629f30 thru 0xc62b33e (0x140f bytes) 2024-09-12T09:31:28.637 INFO:teuthology.orchestra.run.smithi007.stdout:4266 trunc from 0xc9b2683 to 0x14a4762 2024-09-12T09:31:28.663 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:28.658+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:28.692 INFO:teuthology.orchestra.run.smithi007.stdout:4268 write 0x518763e thru 0x5197223 (0xfbe6 bytes) 2024-09-12T09:31:28.693 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.693 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:28.731 INFO:teuthology.orchestra.run.smithi007.stdout:4269 write 0x6fe78f8 thru 0x6fead55 (0x345e bytes) 2024-09-12T09:31:28.741 INFO:teuthology.orchestra.run.smithi007.stdout:4271 write 0x2420f34 thru 0x2423caa (0x2d77 bytes) 2024-09-12T09:31:28.749 INFO:teuthology.orchestra.run.smithi007.stdout:4274 read 0x80a4c9 thru 0x818a0f (0xe547 bytes) 2024-09-12T09:31:28.765 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:28.763+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:28.766 INFO:teuthology.orchestra.run.smithi007.stdout:4275 trunc from 0x6fead56 to 0xe003452 2024-09-12T09:31:28.778 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:28.778 INFO:teuthology.orchestra.run.smithi007.stdout:4278 read 0x222dcce thru 0x223c1e6 (0xe519 bytes) 2024-09-12T09:31:28.779 INFO:teuthology.orchestra.run.smithi007.stdout:4279 punch from 0xae8d55b to 0xae96f0e, (0x99b3 bytes) 2024-09-12T09:31:28.779 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:28.779 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:28.779 INFO:teuthology.orchestra.run.smithi007.stdout:4288 writesame 0x22eae8d thru 0x22ef00a data_size 0x2(0x417e bytes) 2024-09-12T09:31:28.791 INFO:teuthology.orchestra.run.smithi007.stdout:4290 punch from 0x254a055 to 0x2558dbf, (0xed6a bytes) 2024-09-12T09:31:28.791 INFO:teuthology.orchestra.run.smithi007.stdout:4293 read 0xb9bdb79 thru 0xb9bef6c (0x13f4 bytes) 2024-09-12T09:31:28.806 INFO:teuthology.orchestra.run.smithi007.stdout:4295 write 0xec5ae08 thru 0xec5cff7 (0x21f0 bytes) 2024-09-12T09:31:28.813 INFO:teuthology.orchestra.run.smithi007.stdout:4296 write 0x401a07f thru 0x402967b (0xf5fd bytes) 2024-09-12T09:31:28.831 INFO:teuthology.orchestra.run.smithi007.stdout:4297 writesame 0x23d62be thru 0x23d8c4f data_size 0x11(0x2992 bytes) 2024-09-12T09:31:28.840 INFO:teuthology.orchestra.run.smithi007.stdout:4298 punch from 0xe05865e to 0xe05f889, (0x722b bytes) 2024-09-12T09:31:28.840 INFO:teuthology.orchestra.run.smithi007.stdout:4300 punch from 0x77d0a03 to 0x77d1688, (0xc85 bytes) 2024-09-12T09:31:28.840 INFO:teuthology.orchestra.run.smithi007.stdout:4301 write 0x19d6d2f thru 0x19e07f8 (0x9aca bytes) 2024-09-12T09:31:28.857 INFO:teuthology.orchestra.run.smithi007.stdout:4303 punch from 0x31a4b24 to 0x31b39ef, (0xeecb bytes) 2024-09-12T09:31:28.857 INFO:teuthology.orchestra.run.smithi007.stdout:4304 punch from 0x8cdb20f to 0x8ce2b6b, (0x795c bytes) 2024-09-12T09:31:28.857 INFO:teuthology.orchestra.run.smithi007.stdout:4305 write 0x748b8f4 thru 0x74918cc (0x5fd9 bytes) 2024-09-12T09:31:28.865 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:28.863+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:28.868 INFO:teuthology.orchestra.run.smithi007.stdout:4307 write 0x6bae16e thru 0x6bae4e2 (0x375 bytes) 2024-09-12T09:31:28.874 INFO:teuthology.orchestra.run.smithi007.stdout:4309 write 0x1b6402a thru 0x1b6a133 (0x610a bytes) 2024-09-12T09:31:28.880 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:28.880 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:28.882 INFO:teuthology.orchestra.run.smithi007.stdout:4311 read 0x424d9a5 thru 0x424ffd5 (0x2631 bytes) 2024-09-12T09:31:28.882 INFO:teuthology.orchestra.run.smithi007.stdout:4312 write 0xc4b86b5 thru 0xc4c0bed (0x8539 bytes) 2024-09-12T09:31:28.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:28.899 INFO:teuthology.orchestra.run.smithi007.stdout:4313 read 0x4dc06e9 thru 0x4dcf4b1 (0xedc9 bytes) 2024-09-12T09:31:28.899 INFO:teuthology.orchestra.run.smithi007.stdout:4314 read 0xced2bc1 thru 0xced9022 (0x6462 bytes) 2024-09-12T09:31:28.899 INFO:teuthology.orchestra.run.smithi007.stdout:4315 write 0xd4b1849 thru 0xd4ba19c (0x8954 bytes) 2024-09-12T09:31:28.919 INFO:teuthology.orchestra.run.smithi007.stdout:4316 punch from 0x4ee851d to 0x4eef3b6, (0x6e99 bytes) 2024-09-12T09:31:28.919 INFO:teuthology.orchestra.run.smithi007.stdout:4317 write 0xbd58446 thru 0xbd6306d (0xac28 bytes) 2024-09-12T09:31:28.935 INFO:teuthology.orchestra.run.smithi007.stdout:4318 write 0x86ee4ef thru 0x86f6fea (0x8afc bytes) 2024-09-12T09:31:28.948 INFO:teuthology.orchestra.run.smithi007.stdout:4320 write 0x12c94b8 thru 0x12d08e7 (0x7430 bytes) 2024-09-12T09:31:28.953 INFO:teuthology.orchestra.run.smithi007.stdout:4321 writesame 0x7a19b41 thru 0x7a1c08c data_size 0x3e(0x254c bytes) 2024-09-12T09:31:28.962 INFO:teuthology.orchestra.run.smithi007.stdout:4322 punch from 0xf433c3 to 0xf46d5d, (0x399a bytes) 2024-09-12T09:31:28.962 INFO:teuthology.orchestra.run.smithi007.stdout:4324 read 0x1c2389f thru 0x1c2d5dd (0x9d3f bytes) 2024-09-12T09:31:28.962 INFO:teuthology.orchestra.run.smithi007.stdout:4325 read 0xd24201d thru 0xd246b5b (0x4b3f bytes) 2024-09-12T09:31:28.962 INFO:teuthology.orchestra.run.smithi007.stdout:4326 writesame 0x5c0df1c thru 0x5c13850 data_size 0x1(0x5935 bytes) 2024-09-12T09:31:28.966 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:28.963+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:28.983 INFO:teuthology.orchestra.run.smithi007.stdout:4328 read 0x4e20d50 thru 0x4e24004 (0x32b5 bytes) 2024-09-12T09:31:28.983 INFO:teuthology.orchestra.run.smithi007.stdout:4332 write 0x53b1f35 thru 0x53c0661 (0xe72d bytes) 2024-09-12T09:31:29.005 INFO:teuthology.orchestra.run.smithi007.stdout:4333 punch from 0x31363df to 0x313fb22, (0x9743 bytes) 2024-09-12T09:31:29.005 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.006 INFO:teuthology.orchestra.run.smithi007.stdout:4335 write 0xddc6f2e thru 0xddcdcbd (0x6d90 bytes) 2024-09-12T09:31:29.018 INFO:teuthology.orchestra.run.smithi007.stdout:4336 punch from 0x3deba02 to 0x3dfb5cf, (0xfbcd bytes) 2024-09-12T09:31:29.042 INFO:teuthology.orchestra.run.smithi007.stdout:4337 trunc from 0xec5cff8 to 0xbecc38 2024-09-12T09:31:29.044 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:29.044 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:29.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:29.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:29.068 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:29.062+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:29.153 INFO:teuthology.orchestra.run.smithi007.stdout:4338 read 0x9652a9 thru 0x96db07 (0x885f bytes) 2024-09-12T09:31:29.168 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:29.167+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:29.175 INFO:teuthology.orchestra.run.smithi007.stdout:4339 write 0x95ab95f thru 0x95b2bbb (0x725d bytes) 2024-09-12T09:31:29.194 INFO:teuthology.orchestra.run.smithi007.stdout:4341 writesame 0x9ffc196 thru 0xa00b315 data_size 0x70(0xf180 bytes) 2024-09-12T09:31:29.194 INFO:teuthology.orchestra.run.smithi007.stdout:4342 read 0xe676b3 thru 0xe6ed6e (0x76bc bytes) 2024-09-12T09:31:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:4347 read 0x7f4e3ac thru 0x7f5be02 (0xda57 bytes) 2024-09-12T09:31:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:4348 read 0x3c6a1db thru 0x3c6b062 (0xe88 bytes) 2024-09-12T09:31:29.195 INFO:teuthology.orchestra.run.smithi007.stdout:4350 read 0x3411151 thru 0x341ecca (0xdb7a bytes) 2024-09-12T09:31:29.207 INFO:teuthology.orchestra.run.smithi007.stdout:4351 trunc from 0xa00b316 to 0x14f4711 2024-09-12T09:31:29.225 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.225 INFO:teuthology.orchestra.run.smithi007.stdout:4353 read 0x1476c14 thru 0x1482846 (0xbc33 bytes) 2024-09-12T09:31:29.226 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.232 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:29.240 INFO:teuthology.orchestra.run.smithi007.stdout:4355 writesame 0x3edbe1d thru 0x3edc1a2 data_size 0x2(0x386 bytes) 2024-09-12T09:31:29.240 INFO:teuthology.orchestra.run.smithi007.stdout:4356 write 0x2431f31 thru 0x2432939 (0xa09 bytes) 2024-09-12T09:31:29.246 INFO:teuthology.orchestra.run.smithi007.stdout:4357 read 0x21f729a thru 0x21fd7fc (0x6563 bytes) 2024-09-12T09:31:29.246 INFO:teuthology.orchestra.run.smithi007.stdout:4358 punch from 0x349ea83 to 0x349ec94, (0x211 bytes) 2024-09-12T09:31:29.254 INFO:teuthology.orchestra.run.smithi007.stdout:4359 trunc from 0x3edc1a3 to 0x9b47709 2024-09-12T09:31:29.257 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:29.257 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:4360 write 0x756c67 thru 0x75cb87 (0x5f21 bytes) 2024-09-12T09:31:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:29.269 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:29.267+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:29.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:29.276 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:29.280 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.280 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.280 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.280 INFO:teuthology.orchestra.run.smithi007.stdout:4364 read 0x191bfb1 thru 0x19212df (0x532f bytes) 2024-09-12T09:31:29.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:29.293 INFO:teuthology.orchestra.run.smithi007.stdout:4365 write 0xab56906 thru 0xab587da (0x1ed5 bytes) 2024-09-12T09:31:29.293 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:29.294 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:29.300 INFO:teuthology.orchestra.run.smithi007.stdout:4366 writesame 0x61a28a4 thru 0x61a9dd5 data_size 0x7(0x7532 bytes) 2024-09-12T09:31:29.312 INFO:teuthology.orchestra.run.smithi007.stdout:4368 read 0x2e0dccc thru 0x2e122a3 (0x45d8 bytes) 2024-09-12T09:31:29.312 INFO:teuthology.orchestra.run.smithi007.stdout:4369 write 0x7307bc7 thru 0x731615d (0xe597 bytes) 2024-09-12T09:31:29.322 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:29.328 INFO:teuthology.orchestra.run.smithi007.stdout:4371 write 0x2796a67 thru 0x279e6b1 (0x7c4b bytes) 2024-09-12T09:31:29.341 INFO:teuthology.orchestra.run.smithi007.stdout:4372 read 0x1e0dc64 thru 0x1e128c9 (0x4c66 bytes) 2024-09-12T09:31:29.342 INFO:teuthology.orchestra.run.smithi007.stdout:4373 read 0x794af5f thru 0x7955847 (0xa8e9 bytes) 2024-09-12T09:31:29.342 INFO:teuthology.orchestra.run.smithi007.stdout:4375 write 0x22316fc thru 0x2235ada (0x43df bytes) 2024-09-12T09:31:29.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:29.351 INFO:teuthology.orchestra.run.smithi007.stdout:4376 read 0x7b9e3b4 thru 0x7badcdf (0xf92c bytes) 2024-09-12T09:31:29.351 INFO:teuthology.orchestra.run.smithi007.stdout:4377 read 0x5e237c5 thru 0x5e2ec2d (0xb469 bytes) 2024-09-12T09:31:29.361 INFO:teuthology.orchestra.run.smithi007.stdout:4378 writesame 0xbb1956e thru 0xbb1a735 data_size 0x4(0x11c8 bytes) 2024-09-12T09:31:29.361 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.362 INFO:teuthology.orchestra.run.smithi007.stdout:4380 punch from 0x8de020b to 0x8de6268, (0x605d bytes) 2024-09-12T09:31:29.362 INFO:teuthology.orchestra.run.smithi007.stdout:4383 read 0x1431054 thru 0x143c367 (0xb314 bytes) 2024-09-12T09:31:29.362 INFO:teuthology.orchestra.run.smithi007.stdout:4384 read 0x27c33e1 thru 0x27c40a0 (0xcc0 bytes) 2024-09-12T09:31:29.370 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:29.367+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:29.371 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:29.372 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:29.373 INFO:teuthology.orchestra.run.smithi007.stdout:4385 trunc from 0xbb1a736 to 0x3db6cac 2024-09-12T09:31:29.395 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:29.395 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:29.411 INFO:teuthology.orchestra.run.smithi007.stdout:4387 trunc from 0x3db6cac to 0x9d5583e 2024-09-12T09:31:29.421 INFO:teuthology.orchestra.run.smithi007.stdout:4388 read 0x9110762 thru 0x9111a59 (0x12f8 bytes) 2024-09-12T09:31:29.424 INFO:teuthology.orchestra.run.smithi007.stdout:4389 trunc from 0x9d5583e to 0xba20f29 2024-09-12T09:31:29.424 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:29.425 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:29.425 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:29.434 INFO:teuthology.orchestra.run.smithi007.stdout:4390 read 0x8ee413c thru 0x8ee7b03 (0x39c8 bytes) 2024-09-12T09:31:29.434 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.441 INFO:teuthology.orchestra.run.smithi007.stdout:4394 trunc from 0xba20f29 to 0x683261c 2024-09-12T09:31:29.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:29.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.471 INFO:teuthology.orchestra.run.smithi007.stdout:4396 writesame 0xbad4893 thru 0xbae4666 data_size 0x14(0xfdd4 bytes) 2024-09-12T09:31:29.471 INFO:teuthology.orchestra.run.smithi007.stdout:4397 write 0x89ca287 thru 0x89d212e (0x7ea8 bytes) 2024-09-12T09:31:29.472 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:29.466+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:4401 punch from 0x47aa5ab to 0x47ad43c, (0x2e91 bytes) 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.484 INFO:teuthology.orchestra.run.smithi007.stdout:4406 write 0x48ab157 thru 0x48ac3fd (0x12a7 bytes) 2024-09-12T09:31:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.491 INFO:teuthology.orchestra.run.smithi007.stdout:4408 writesame 0x6ad7a7 thru 0x6b80df data_size 0x1(0xa939 bytes) 2024-09-12T09:31:29.555 INFO:teuthology.orchestra.run.smithi007.stdout:4410 write 0xd3ef884 thru 0xd3f117d (0x18fa bytes) 2024-09-12T09:31:29.564 INFO:teuthology.orchestra.run.smithi007.stdout:4411 trunc from 0xd3f117e to 0xe7096fb 2024-09-12T09:31:29.572 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:29.571+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:29.575 INFO:teuthology.orchestra.run.smithi007.stdout:4412 write 0xe1fadfc thru 0xe207acb (0xccd0 bytes) 2024-09-12T09:31:29.581 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.144174658000001, 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.144149426999999, 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.144111507000002, 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.144081505999999, 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.136047085000001, 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.135726212000002, 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.135302385999999, 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.135250907, 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.135222159000001, 2024-09-12T09:31:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.134671693000001, 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.134628871, 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.134582980000001, 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.134071422000002, 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.134023642999999, 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.133967996999999, 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.133935976, 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.133371827000001, 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.132977925999999, 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.132586228000001, 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7647201099999998, 2024-09-12T09:31:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:31:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:29.700 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:29.702 INFO:teuthology.orchestra.run.smithi007.stdout:4413 write 0x44f61f4 thru 0x44f7c6c (0x1a79 bytes) 2024-09-12T09:31:29.702 INFO:teuthology.orchestra.run.smithi007.stdout:4414 read 0xa188bb7 thru 0xa18a437 (0x1881 bytes) 2024-09-12T09:31:29.702 INFO:teuthology.orchestra.run.smithi007.stdout:4416 read 0x75588bb thru 0x7566165 (0xd8ab bytes) 2024-09-12T09:31:29.702 INFO:teuthology.orchestra.run.smithi007.stdout:4418 write 0xb2bf7ef thru 0xb2c596c (0x617e bytes) 2024-09-12T09:31:29.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4419 punch from 0x1a2dc64 to 0x1a3b7eb, (0xdb87 bytes) 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4420 write 0x38f78f1 thru 0x38f97db (0x1eeb bytes) 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4421 write 0xd0a506a thru 0xd0b16bf (0xc656 bytes) 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4422 read 0x3e3e8d3 thru 0x3e4d656 (0xed84 bytes) 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4423 writesame 0x8aa0a1a thru 0x8ab04e4 data_size 0x1(0xfacb bytes) 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.704 INFO:teuthology.orchestra.run.smithi007.stdout:4425 trunc from 0xe7096fb to 0x4f0b321 2024-09-12T09:31:29.707 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:29.702+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:29.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:31:29.725 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.725 INFO:teuthology.orchestra.run.smithi007.stdout:4427 write 0x33f792e thru 0x33fa9a6 (0x3079 bytes) 2024-09-12T09:31:29.733 INFO:teuthology.orchestra.run.smithi007.stdout:4428 punch from 0x20f6971 to 0x2103b9c, (0xd22b bytes) 2024-09-12T09:31:29.734 INFO:teuthology.orchestra.run.smithi007.stdout:4429 punch from 0x36ca01b to 0x36d6a3f, (0xca24 bytes) 2024-09-12T09:31:29.755 INFO:teuthology.orchestra.run.smithi007.stdout:4430 writesame 0xa227579 thru 0xa235825 data_size 0x45(0xe2ad bytes) 2024-09-12T09:31:29.755 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.755 INFO:teuthology.orchestra.run.smithi007.stdout:4432 write 0x885b7f6 thru 0x88653d4 (0x9bdf bytes) 2024-09-12T09:31:29.771 INFO:teuthology.orchestra.run.smithi007.stdout:4433 punch from 0x6163ef4 to 0x616f47e, (0xb58a bytes) 2024-09-12T09:31:29.771 INFO:teuthology.orchestra.run.smithi007.stdout:4434 read 0x8fb1f84 thru 0x8fb864e (0x66cb bytes) 2024-09-12T09:31:29.771 INFO:teuthology.orchestra.run.smithi007.stdout:4435 read 0x345de83 thru 0x3464f90 (0x710e bytes) 2024-09-12T09:31:29.771 INFO:teuthology.orchestra.run.smithi007.stdout:4437 write 0x3fd7c8 thru 0x403e44 (0x667d bytes) 2024-09-12T09:31:29.793 INFO:teuthology.orchestra.run.smithi007.stdout:4438 writesame 0x3cb5141 thru 0x3cbd11f data_size 0x1(0x7fdf bytes) 2024-09-12T09:31:29.803 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:29.798+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:29.813 INFO:teuthology.orchestra.run.smithi007.stdout:4440 read 0x3367134 thru 0x336eb12 (0x79df bytes) 2024-09-12T09:31:29.814 INFO:teuthology.orchestra.run.smithi007.stdout:4441 punch from 0x55bb801 to 0x55c4a8f, (0x928e bytes) 2024-09-12T09:31:29.814 INFO:teuthology.orchestra.run.smithi007.stdout:4442 read 0x8860b2e thru 0x88685c1 (0x7a94 bytes) 2024-09-12T09:31:29.815 INFO:teuthology.orchestra.run.smithi007.stdout:4443 writesame 0x4b2a364 thru 0x4b303c0 data_size 0x3(0x605d bytes) 2024-09-12T09:31:29.828 INFO:teuthology.orchestra.run.smithi007.stdout:4444 read 0x9acc091 thru 0x9ad59af (0x991f bytes) 2024-09-12T09:31:29.828 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.828 INFO:teuthology.orchestra.run.smithi007.stdout:4448 read 0x6399c95 thru 0x63a47d1 (0xab3d bytes) 2024-09-12T09:31:29.829 INFO:teuthology.orchestra.run.smithi007.stdout:4449 write 0x6f6545e thru 0x6f703fe (0xafa1 bytes) 2024-09-12T09:31:29.844 INFO:teuthology.orchestra.run.smithi007.stdout:4451 read 0x32956dc thru 0x32a489e (0xf1c3 bytes) 2024-09-12T09:31:29.845 INFO:teuthology.orchestra.run.smithi007.stdout:4452 read 0x1211c84 thru 0x121e41f (0xc79c bytes) 2024-09-12T09:31:29.845 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:29.850 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:29.850 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:29.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:29.866 INFO:teuthology.orchestra.run.smithi007.stdout:4455 trunc from 0xa235826 to 0x373f68 2024-09-12T09:31:29.910 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:29.906+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:29.959 INFO:teuthology.orchestra.run.smithi007.stdout:4456 write 0xb3e2868 thru 0xb3ef1c5 (0xc95e bytes) 2024-09-12T09:31:29.974 INFO:teuthology.orchestra.run.smithi007.stdout:4460 write 0x5c69f3f thru 0x5c6be13 (0x1ed5 bytes) 2024-09-12T09:31:29.981 INFO:teuthology.orchestra.run.smithi007.stdout:4461 write 0x801700e thru 0x801b8bf (0x48b2 bytes) 2024-09-12T09:31:30.001 INFO:teuthology.orchestra.run.smithi007.stdout:4462 trunc from 0xb3ef1c6 to 0x234d0c8 2024-09-12T09:31:30.004 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:30.002+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:30.022 INFO:teuthology.orchestra.run.smithi007.stdout:4463 read 0x139a6bf thru 0x139f05e (0x49a0 bytes) 2024-09-12T09:31:30.022 INFO:teuthology.orchestra.run.smithi007.stdout:4464 read 0x138c0b7 thru 0x1395130 (0x907a bytes) 2024-09-12T09:31:30.034 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:30.034 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:30.046 INFO:teuthology.orchestra.run.smithi007.stdout:4465 write 0xbf43f1b thru 0xbf4a131 (0x6217 bytes) 2024-09-12T09:31:30.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:30.059 INFO:teuthology.orchestra.run.smithi007.stdout:4466 read 0x8563799 thru 0x85648a8 (0x1110 bytes) 2024-09-12T09:31:30.059 INFO:teuthology.orchestra.run.smithi007.stdout:4468 read 0x3922446 thru 0x392fe5c (0xda17 bytes) 2024-09-12T09:31:30.059 INFO:teuthology.orchestra.run.smithi007.stdout:4469 read 0xa4618b thru 0xa49d42 (0x3bb8 bytes) 2024-09-12T09:31:30.060 INFO:teuthology.orchestra.run.smithi007.stdout:4470 writesame 0x8fd3c19 thru 0x8fd5316 data_size 0x1b(0x16fe bytes) 2024-09-12T09:31:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:4471 read 0xb23f01d thru 0xb2443bf (0x53a3 bytes) 2024-09-12T09:31:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.076 INFO:teuthology.orchestra.run.smithi007.stdout:4474 write 0xc5ffff4 thru 0xc60acf5 (0xad02 bytes) 2024-09-12T09:31:30.093 INFO:teuthology.orchestra.run.smithi007.stdout:4475 writesame 0xad0a1ab thru 0xad0cbcc data_size 0x1(0x2a22 bytes) 2024-09-12T09:31:30.105 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.105 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.105 INFO:teuthology.orchestra.run.smithi007.stdout:4480 read 0x757ed37 thru 0x75837d3 (0x4a9d bytes) 2024-09-12T09:31:30.106 INFO:teuthology.orchestra.run.smithi007.stdout:4481 write 0x4db342c thru 0x4dbce12 (0x99e7 bytes) 2024-09-12T09:31:30.106 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:30.103+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:30.118 INFO:teuthology.orchestra.run.smithi007.stdout:4482 write 0x6704ef4 thru 0x67124eb (0xd5f8 bytes) 2024-09-12T09:31:30.133 INFO:teuthology.orchestra.run.smithi007.stdout:4483 punch from 0x1e92 to 0x8647, (0x67b5 bytes) 2024-09-12T09:31:30.133 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.133 INFO:teuthology.orchestra.run.smithi007.stdout:4486 punch from 0x85b7ade to 0x85c7759, (0xfc7b bytes) 2024-09-12T09:31:30.134 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.134 INFO:teuthology.orchestra.run.smithi007.stdout:4489 read 0xc23c3b2 thru 0xc2453a7 (0x8ff6 bytes) 2024-09-12T09:31:30.134 INFO:teuthology.orchestra.run.smithi007.stdout:4490 read 0x41ce05d thru 0x41db541 (0xd4e5 bytes) 2024-09-12T09:31:30.134 INFO:teuthology.orchestra.run.smithi007.stdout:4491 read 0x4a5ab48 thru 0x4a6495d (0x9e16 bytes) 2024-09-12T09:31:30.148 INFO:teuthology.orchestra.run.smithi007.stdout:4494 trunc from 0xc60acf6 to 0x5041ced 2024-09-12T09:31:30.179 INFO:teuthology.orchestra.run.smithi007.stdout:4495 punch from 0x44b5908 to 0x44bbca7, (0x639f bytes) 2024-09-12T09:31:30.180 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.200 INFO:teuthology.orchestra.run.smithi007.stdout:4498 write 0x79dafb3 thru 0x79e1b20 (0x6b6e bytes) 2024-09-12T09:31:30.206 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:30.202+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:30.213 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.214 INFO:teuthology.orchestra.run.smithi007.stdout:4501 punch from 0xd970c8 to 0xda356d, (0xc4a5 bytes) 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.932829960999996, 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:30.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:30.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:30.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:30.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.473333732, 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:30.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.099916781000005, 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:30.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:30.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:30.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:30.238 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.099869534000007, 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:30.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:30.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.099576458000001, 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:30.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.099014526000005, 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:30.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:30.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.098963162999993, 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.098390283000001, 2024-09-12T09:31:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:30.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:30.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.098237983999994, 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:30.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.097880893999999, 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.270 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.097255923999995, 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.097204587999997, 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.096409797999996, 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.096334908000003, 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.096282587999994, 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.096233990000002, 2024-09-12T09:31:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.095979525000004, 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.092525234999997, 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.034956687999994, 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.563685450999998, 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.330 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:30.331 INFO:teuthology.orchestra.run.smithi007.stdout:4502 write 0xb82bbf1 thru 0xb82ef47 (0x3357 bytes) 2024-09-12T09:31:30.332 INFO:teuthology.orchestra.run.smithi007.stdout:4504 write 0x856f22e thru 0x85713b5 (0x2188 bytes) 2024-09-12T09:31:30.332 INFO:teuthology.orchestra.run.smithi007.stdout:4505 trunc from 0xb82ef48 to 0xab9a6e7 2024-09-12T09:31:30.332 INFO:teuthology.orchestra.run.smithi007.stdout:4506 trunc from 0xab9a6e7 to 0xe4e480 2024-09-12T09:31:30.333 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:30.333 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:30.326+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:30.388 INFO:teuthology.orchestra.run.smithi007.stdout:4507 write 0xad65cbb thru 0xad65f54 (0x29a bytes) 2024-09-12T09:31:30.395 INFO:teuthology.orchestra.run.smithi007.stdout:4508 write 0x981d587 thru 0x9829ba9 (0xc623 bytes) 2024-09-12T09:31:30.421 INFO:teuthology.orchestra.run.smithi007.stdout:4509 writesame 0x48d7cbc thru 0x48de34d data_size 0x13(0x6692 bytes) 2024-09-12T09:31:30.433 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:30.431+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:30.438 INFO:teuthology.orchestra.run.smithi007.stdout:4510 read 0x4fe9b45 thru 0x4fed1e1 (0x369d bytes) 2024-09-12T09:31:30.438 INFO:teuthology.orchestra.run.smithi007.stdout:4513 read 0x543ac85 thru 0x5441e3d (0x71b9 bytes) 2024-09-12T09:31:30.438 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.438 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.438 INFO:teuthology.orchestra.run.smithi007.stdout:4518 write 0x442d648 thru 0x4433c51 (0x660a bytes) 2024-09-12T09:31:30.451 INFO:teuthology.orchestra.run.smithi007.stdout:4520 writesame 0x247f9c0 thru 0x2482408 data_size 0x5(0x2a49 bytes) 2024-09-12T09:31:30.491 INFO:teuthology.orchestra.run.smithi007.stdout:4521 trunc from 0xad65f55 to 0x7e7bfe 2024-09-12T09:31:30.509 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6] out_osds: [1, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:30.509 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:30.509 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T09:31:30.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:31:30.519 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:30.520 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:30.528 INFO:teuthology.orchestra.run.smithi007.stdout:4522 punch from 0x3b9b82 to 0x3c5420, (0xb89e bytes) 2024-09-12T09:31:30.534 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:30.531+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:30.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:30.547 INFO:teuthology.orchestra.run.smithi007.stdout:4523 write 0x443392b thru 0x4435e17 (0x24ed bytes) 2024-09-12T09:31:30.571 INFO:teuthology.orchestra.run.smithi007.stdout:4524 writesame 0x86e2c75 thru 0x86e51ea data_size 0x23(0x2576 bytes) 2024-09-12T09:31:30.571 INFO:teuthology.orchestra.run.smithi007.stdout:4525 write 0x45f5d1a thru 0x45fe4c9 (0x87b0 bytes) 2024-09-12T09:31:30.582 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.582 INFO:teuthology.orchestra.run.smithi007.stdout:4528 punch from 0xc0a823 to 0xc15913, (0xb0f0 bytes) 2024-09-12T09:31:30.583 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.583 INFO:teuthology.orchestra.run.smithi007.stdout:4531 write 0x6d7b1c0 thru 0x6d843bd (0x91fe bytes) 2024-09-12T09:31:30.595 INFO:teuthology.orchestra.run.smithi007.stdout:4533 trunc from 0x86e51eb to 0x792f688 2024-09-12T09:31:30.621 INFO:teuthology.orchestra.run.smithi007.stdout:4534 write 0xbb49244 thru 0xbb4e80a (0x55c7 bytes) 2024-09-12T09:31:30.634 INFO:teuthology.orchestra.run.smithi007.stdout:4535 writesame 0x8d45105 thru 0x8d4654d data_size 0x3(0x1449 bytes) 2024-09-12T09:31:30.634 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:30.631+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:30.641 INFO:teuthology.orchestra.run.smithi007.stdout:4537 write 0x1ad9439 thru 0x1ae49d6 (0xb59e bytes) 2024-09-12T09:31:30.655 INFO:teuthology.orchestra.run.smithi007.stdout:4538 write 0x58baeae thru 0x58c0aa1 (0x5bf4 bytes) 2024-09-12T09:31:30.665 INFO:teuthology.orchestra.run.smithi007.stdout:4540 write 0x144625f thru 0x1454c95 (0xea37 bytes) 2024-09-12T09:31:30.669 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:30.669 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:30.669 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:30.680 INFO:teuthology.orchestra.run.smithi007.stdout:4541 write 0x877343c thru 0x878245d (0xf022 bytes) 2024-09-12T09:31:30.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:30.697 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.704 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:31:30.712 INFO:teuthology.orchestra.run.smithi007.stdout:4543 writesame 0xbd27715 thru 0xbd2de41 data_size 0x1(0x672d bytes) 2024-09-12T09:31:30.713 INFO:teuthology.orchestra.run.smithi007.stdout:4544 writesame 0xab88b5d thru 0xab8f5de data_size 0x1(0x6a82 bytes) 2024-09-12T09:31:30.732 INFO:teuthology.orchestra.run.smithi007.stdout:4545 punch from 0x1134d0 to 0x116053, (0x2b83 bytes) 2024-09-12T09:31:30.735 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:30.730+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:30.741 INFO:teuthology.orchestra.run.smithi007.stdout:4546 write 0xd0ca0a0 thru 0xd0d205f (0x7fc0 bytes) 2024-09-12T09:31:30.749 INFO:teuthology.orchestra.run.smithi007.stdout:4547 write 0x7b49a49 thru 0x7b4ead8 (0x5090 bytes) 2024-09-12T09:31:30.759 INFO:teuthology.orchestra.run.smithi007.stdout:4548 writesame 0x4c9dcd3 thru 0x4c9ef3b data_size 0x1(0x1269 bytes) 2024-09-12T09:31:30.773 INFO:teuthology.orchestra.run.smithi007.stdout:4551 trunc from 0xd0d2060 to 0x76c86ea 2024-09-12T09:31:30.809 INFO:teuthology.orchestra.run.smithi007.stdout:4552 write 0xb6fd7b4 thru 0xb6fe500 (0xd4d bytes) 2024-09-12T09:31:30.815 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:30.815 INFO:teuthology.orchestra.run.smithi007.stdout:4554 write 0x13cf79f thru 0x13d0b99 (0x13fb bytes) 2024-09-12T09:31:30.822 INFO:teuthology.orchestra.run.smithi007.stdout:4555 punch from 0x1139a0e to 0x1146912, (0xcf04 bytes) 2024-09-12T09:31:30.822 INFO:teuthology.orchestra.run.smithi007.stdout:4556 read 0x9c7a0b4 thru 0x9c85754 (0xb6a1 bytes) 2024-09-12T09:31:30.826 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.41531062999999, 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:30.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:30.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.41521536800001, 2024-09-12T09:31:30.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:30.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:30.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.411530744, 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:30.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.409725654, 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:30.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:30.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.409178979, 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:30.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.40841053600001, 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:30.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:30.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.408000168, 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:30.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:30.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.407860992, 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:30.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:30.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:30.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.400082181, 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:30.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.39771816699999, 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:30.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.386966274, 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:30.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:30.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.386769883, 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:30.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:30.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.385275463, 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:30.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.685032602000007, 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.895 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:30.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.685010184000006, 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.684987043000007, 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:30.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:30.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.684962084000006, 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:30.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.641610569999997, 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:30.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:30.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:30.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.025132528, 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:30.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:30.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.025033238000006, 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:30.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:30.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:30.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:30.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:30.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:30.929 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:30.930 INFO:teuthology.orchestra.run.smithi007.stdout:4558 trunc from 0xb6fe501 to 0x69dbcfb 2024-09-12T09:31:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 119, 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:31:30.700282+0000", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:29:16.082807+0000", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:20.251891+0000", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:31:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:31:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:31:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:31:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "118", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 43, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 118, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 103, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:31:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 66, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 115, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 116, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 115, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 111, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:31:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 115, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:31:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:31:30.993 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:30.994 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 111, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:31:30.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:30.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:31:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.001 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:31.003 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.004 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:31:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.2", 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1a", 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:31:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:31:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:31.011 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:31.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:31.013 INFO:teuthology.orchestra.run.smithi007.stdout:4559 writesame 0xbaeb182 thru 0xbaf18f5 data_size 0x6(0x6774 bytes) 2024-09-12T09:31:31.013 INFO:teuthology.orchestra.run.smithi007.stdout:4560 trunc from 0xbaf18f6 to 0xb6341d7 2024-09-12T09:31:31.013 INFO:teuthology.orchestra.run.smithi007.stdout:4562 write 0x655f3d4 thru 0x6561fc1 (0x2bee bytes) 2024-09-12T09:31:31.013 INFO:teuthology.orchestra.run.smithi007.stdout:4565 read 0x29fcb17 thru 0x2a014cf (0x49b9 bytes) 2024-09-12T09:31:31.013 INFO:teuthology.orchestra.run.smithi007.stdout:4566 writesame 0x28360d8 thru 0x28408b6 data_size 0x9(0xa7df bytes) 2024-09-12T09:31:31.015 INFO:tasks.thrashosds.thrasher:j is {'epoch': 119, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:31:30.700282+0000', 'last_up_change': '2024-09-12T09:29:16.082807+0000', 'last_in_change': '2024-09-12T09:31:20.251891+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 111, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [6, 7]}, {'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:31:31.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:4567 write 0xc2bcc0 thru 0xc2c757 (0xa98 bytes) 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:4568 read 0x1fb5632 thru 0x1fc13de (0xbdad bytes) 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:4569 read 0x7a4cd66 thru 0x7a5cbd3 (0xfe6e bytes) 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:4570 writesame 0x1dd698e thru 0x1de2b7c data_size 0x43(0xc1ef bytes) 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.016 INFO:teuthology.orchestra.run.smithi007.stdout:4574 read 0x5981b7d thru 0x59907ae (0xec32 bytes) 2024-09-12T09:31:31.016 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:30.927+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:31.017 INFO:teuthology.orchestra.run.smithi007.stdout:4575 write 0x9fcbf61 thru 0x9fd9c09 (0xdca9 bytes) 2024-09-12T09:31:31.017 INFO:teuthology.orchestra.run.smithi007.stdout:4576 trunc from 0xb6341d7 to 0x9745888 2024-09-12T09:31:31.017 INFO:teuthology.orchestra.run.smithi007.stdout:4577 read 0x20caebc thru 0x20d2328 (0x746d bytes) 2024-09-12T09:31:31.018 INFO:teuthology.orchestra.run.smithi007.stdout:4579 write 0x8628ccd thru 0x86351f7 (0xc52b bytes) 2024-09-12T09:31:31.018 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.018 INFO:teuthology.orchestra.run.smithi007.stdout:4581 punch from 0x20ac2de to 0x20b9926, (0xd648 bytes) 2024-09-12T09:31:31.018 INFO:teuthology.orchestra.run.smithi007.stdout:4582 write 0x7bd3125 thru 0x7bd68ed (0x37c9 bytes) 2024-09-12T09:31:31.026 INFO:teuthology.orchestra.run.smithi007.stdout:4583 read 0x38004f8 thru 0x380ee47 (0xe950 bytes) 2024-09-12T09:31:31.048 INFO:teuthology.orchestra.run.smithi007.stdout:4584 write 0xc7b50af thru 0xc7b9314 (0x4266 bytes) 2024-09-12T09:31:31.049 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:31.047+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:31.058 INFO:teuthology.orchestra.run.smithi007.stdout:4585 write 0xb02fce7 thru 0xb03b09b (0xb3b5 bytes) 2024-09-12T09:31:31.074 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.075 INFO:teuthology.orchestra.run.smithi007.stdout:4587 read 0x335dd53 thru 0x33652f7 (0x75a5 bytes) 2024-09-12T09:31:31.075 INFO:teuthology.orchestra.run.smithi007.stdout:4588 writesame 0x82247ac thru 0x822ddad data_size 0x5b(0x9602 bytes) 2024-09-12T09:31:31.100 INFO:teuthology.orchestra.run.smithi007.stdout:4590 trunc from 0xc7b9315 to 0x4dd3385 2024-09-12T09:31:31.126 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.132 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:31.130+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:31.141 INFO:teuthology.orchestra.run.smithi007.stdout:4592 write 0x7881df9 thru 0x7891261 (0xf469 bytes) 2024-09-12T09:31:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.164 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.184 INFO:teuthology.orchestra.run.smithi007.stdout:4596 write 0xcff7b7c thru 0xd004583 (0xca08 bytes) 2024-09-12T09:31:31.204 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:31.204 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:31.204 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:31.204 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:31.213 INFO:teuthology.orchestra.run.smithi007.stdout:4597 trunc from 0xd004584 to 0x475d7ed 2024-09-12T09:31:31.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:31.233 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:31.231+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:31.255 INFO:teuthology.orchestra.run.smithi007.stdout:4599 write 0xcc478c1 thru 0xcc4cce7 (0x5427 bytes) 2024-09-12T09:31:31.266 INFO:teuthology.orchestra.run.smithi007.stdout:4600 read 0x3e0dd6a thru 0x3e117a8 (0x3a3f bytes) 2024-09-12T09:31:31.279 INFO:teuthology.orchestra.run.smithi007.stdout:4601 trunc from 0xcc4cce8 to 0x445607a 2024-09-12T09:31:31.303 INFO:teuthology.orchestra.run.smithi007.stdout:4602 read 0x16bbc0f thru 0x16bfdd9 (0x41cb bytes) 2024-09-12T09:31:31.326 INFO:teuthology.orchestra.run.smithi007.stdout:4603 write 0xae13d02 thru 0xae17b6e (0x3e6d bytes) 2024-09-12T09:31:31.334 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:31.331+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:31.338 INFO:teuthology.orchestra.run.smithi007.stdout:4604 read 0x9f2c1e5 thru 0x9f2f636 (0x3452 bytes) 2024-09-12T09:31:31.339 INFO:teuthology.orchestra.run.smithi007.stdout:4605 read 0x5483a20 thru 0x5484c3e (0x121f bytes) 2024-09-12T09:31:31.339 INFO:teuthology.orchestra.run.smithi007.stdout:4608 read 0x7a76321 thru 0x7a82d1f (0xc9ff bytes) 2024-09-12T09:31:31.339 INFO:teuthology.orchestra.run.smithi007.stdout:4609 read 0x4c35667 thru 0x4c3d396 (0x7d30 bytes) 2024-09-12T09:31:31.351 INFO:teuthology.orchestra.run.smithi007.stdout:4612 writesame 0xbbfa0a6 thru 0xbbfe9b2 data_size 0x1(0x490d bytes) 2024-09-12T09:31:31.351 INFO:teuthology.orchestra.run.smithi007.stdout:4614 punch from 0x31e8e00 to 0x31ee3a3, (0x55a3 bytes) 2024-09-12T09:31:31.351 INFO:teuthology.orchestra.run.smithi007.stdout:4615 write 0x24d2c7e thru 0x24e29ed (0xfd70 bytes) 2024-09-12T09:31:31.380 INFO:teuthology.orchestra.run.smithi007.stdout:4616 write 0x4ea0dcb thru 0x4ea1ffa (0x1230 bytes) 2024-09-12T09:31:31.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:31.386 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:31.390 INFO:teuthology.orchestra.run.smithi007.stdout:4617 trunc from 0xbbfe9b3 to 0x97cd940 2024-09-12T09:31:31.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:31.407 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-12T09:31:30.704792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1675248794,"num_objects":1654,"num_object_clones":424,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":593,"num_read":208163,"num_read_kb":1373283,"num_write":14307,"num_write_kb":765540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2557,"num_bytes_recovered":617056731,"num_keys_recovered":482,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11092,"ondisk_log_size":11092,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":303576,"kb_used_data":88588,"kb_used_omap":9,"kb_used_meta":214774,"kb_avail":565927464,"statfs":{"total":579820584960,"available":579509723136,"internally_reserved":0,"allocated":90714112,"data_stored":1083468880,"data_compressed":16476169,"data_compressed_allocated":72445952,"data_compressed_original":1073221169,"omap_allocated":9562,"internal_metadata":219929254},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":220728327,"num_objects":187,"num_object_clones":49,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":69,"num_read":15357,"num_read_kb":82495,"num_write":1234,"num_write_kb":47879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":51177368,"num_keys_recovered":170,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.028336"},"pg_stats":[{"pgid":"3.1f","version":"113'608","reported_seq":5756,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.768589+0000","last_change":"2024-09-12T09:31:12.221541+0000","last_active":"2024-09-12T09:31:25.768589+0000","last_peered":"2024-09-12T09:31:25.768589+0000","last_clean":"2024-09-12T09:31:25.768589+0000","last_became_active":"2024-09-12T09:31:12.219747+0000","last_became_peered":"2024-09-12T09:31:12.219747+0000","last_unstale":"2024-09-12T09:31:25.768589+0000","last_undegraded":"2024-09-12T09:31:25.768589+0000","last_fullsized":"2024-09-12T09:31:25.768589+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:48:22.968982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":93061299,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":7385,"num_read_kb":92625,"num_write":824,"num_write_kb":47163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":37755293,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"110'472","reported_seq":4706,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.787317+0000","last_change":"2024-09-12T09:31:25.691962+0000","last_active":"2024-09-12T09:31:25.787317+0000","last_peered":"2024-09-12T09:31:25.787317+0000","last_clean":"2024-09-12T09:31:25.787317+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:31:25.787317+0000","last_undegraded":"2024-09-12T09:31:25.787317+0000","last_fullsized":"2024-09-12T09:31:25.787317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:32.014697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":44777541,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":6698,"num_read_kb":34834,"num_write":616,"num_write_kb":33938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"114'259","reported_seq":6863,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.682735+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:31:25.682735+0000","last_peered":"2024-09-12T09:31:25.682735+0000","last_clean":"2024-09-12T09:31:25.682735+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:31:25.682735+0000","last_undegraded":"2024-09-12T09:31:25.682735+0000","last_fullsized":"2024-09-12T09:31:25.682735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:22.020651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":54266104,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":7363,"num_read_kb":75298,"num_write":252,"num_write_kb":41010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"116'317","reported_seq":4324,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.759404+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:31:25.759404+0000","last_peered":"2024-09-12T09:31:25.759404+0000","last_clean":"2024-09-12T09:31:25.759404+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:31:25.759404+0000","last_undegraded":"2024-09-12T09:31:25.759404+0000","last_fullsized":"2024-09-12T09:31:25.759404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:28.714244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":24791620,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":4976,"num_read_kb":35016,"num_write":372,"num_write_kb":12174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"117'254","reported_seq":5289,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.744508+0000","last_change":"2024-09-12T09:31:12.230274+0000","last_active":"2024-09-12T09:31:26.744508+0000","last_peered":"2024-09-12T09:31:26.744508+0000","last_clean":"2024-09-12T09:31:26.744508+0000","last_became_active":"2024-09-12T09:31:12.228841+0000","last_became_peered":"2024-09-12T09:31:12.228841+0000","last_unstale":"2024-09-12T09:31:26.744508+0000","last_undegraded":"2024-09-12T09:31:26.744508+0000","last_fullsized":"2024-09-12T09:31:26.744508+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:13.438014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":35556357,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":5533,"num_read_kb":17410,"num_write":267,"num_write_kb":5770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":4210634,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"117'400","reported_seq":7293,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.849614+0000","last_change":"2024-09-12T09:30:46.979355+0000","last_active":"2024-09-12T09:31:26.849614+0000","last_peered":"2024-09-12T09:31:26.849614+0000","last_clean":"2024-09-12T09:31:26.849614+0000","last_became_active":"2024-09-12T09:30:46.978472+0000","last_became_peered":"2024-09-12T09:30:46.978472+0000","last_unstale":"2024-09-12T09:31:26.849614+0000","last_undegraded":"2024-09-12T09:31:26.849614+0000","last_fullsized":"2024-09-12T09:31:26.849614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:00:25.494231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":42179373,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":7529,"num_read_kb":27714,"num_write":466,"num_write_kb":36523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2195003,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"116'240","reported_seq":2668,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.724395+0000","last_change":"2024-09-12T09:30:44.080862+0000","last_active":"2024-09-12T09:31:25.724395+0000","last_peered":"2024-09-12T09:31:25.724395+0000","last_clean":"2024-09-12T09:31:25.724395+0000","last_became_active":"2024-09-12T09:30:43.967238+0000","last_became_peered":"2024-09-12T09:30:43.967238+0000","last_unstale":"2024-09-12T09:31:25.724395+0000","last_undegraded":"2024-09-12T09:31:25.724395+0000","last_fullsized":"2024-09-12T09:31:25.724395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:15.157170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":30245256,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":12,"num_read":2916,"num_read_kb":9081,"num_write":301,"num_write_kb":4066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":970649,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"100'260","reported_seq":2733,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.684447+0000","last_change":"2024-09-12T09:31:12.214934+0000","last_active":"2024-09-12T09:31:25.684447+0000","last_peered":"2024-09-12T09:31:25.684447+0000","last_clean":"2024-09-12T09:31:25.684447+0000","last_became_active":"2024-09-12T09:31:12.214331+0000","last_became_peered":"2024-09-12T09:31:12.214331+0000","last_unstale":"2024-09-12T09:31:25.684447+0000","last_undegraded":"2024-09-12T09:31:25.684447+0000","last_fullsized":"2024-09-12T09:31:25.684447+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:52:16.673413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":16701240,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":21,"num_read":1802,"num_read_kb":7160,"num_write":314,"num_write_kb":5825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":4452429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"117'237","reported_seq":3006,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.901905+0000","last_change":"2024-09-12T09:31:12.221774+0000","last_active":"2024-09-12T09:31:25.901905+0000","last_peered":"2024-09-12T09:31:25.901905+0000","last_clean":"2024-09-12T09:31:25.901905+0000","last_became_active":"2024-09-12T09:31:12.220032+0000","last_became_peered":"2024-09-12T09:31:12.220032+0000","last_unstale":"2024-09-12T09:31:25.901905+0000","last_undegraded":"2024-09-12T09:31:25.901905+0000","last_fullsized":"2024-09-12T09:31:25.901905+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:29:40.293228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":55733917,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":12,"num_read":3271,"num_read_kb":50633,"num_write":345,"num_write_kb":39158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"110'319","reported_seq":14132,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.682641+0000","last_change":"2024-09-12T09:30:33.669800+0000","last_active":"2024-09-12T09:31:25.682641+0000","last_peered":"2024-09-12T09:31:25.682641+0000","last_clean":"2024-09-12T09:31:25.682641+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:31:25.682641+0000","last_undegraded":"2024-09-12T09:31:25.682641+0000","last_fullsized":"2024-09-12T09:31:25.682641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:59.629201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":67618636,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":14223,"num_read_kb":71443,"num_write":408,"num_write_kb":29219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"117'599","reported_seq":7871,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.571830+0000","last_change":"2024-09-12T09:30:34.672921+0000","last_active":"2024-09-12T09:31:26.571830+0000","last_peered":"2024-09-12T09:31:26.571830+0000","last_clean":"2024-09-12T09:31:26.571830+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:31:26.571830+0000","last_undegraded":"2024-09-12T09:31:26.571830+0000","last_fullsized":"2024-09-12T09:31:26.571830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:11:06.356470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":105452496,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":8694,"num_read_kb":97926,"num_write":850,"num_write_kb":43875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"110'657","reported_seq":11573,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.713264+0000","last_change":"2024-09-12T09:30:33.668610+0000","last_active":"2024-09-12T09:31:25.713264+0000","last_peered":"2024-09-12T09:31:25.713264+0000","last_clean":"2024-09-12T09:31:25.713264+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:31:25.713264+0000","last_undegraded":"2024-09-12T09:31:25.713264+0000","last_fullsized":"2024-09-12T09:31:25.713264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:58:43.024244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":48654372,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":15,"num_read":12877,"num_read_kb":48977,"num_write":994,"num_write_kb":28898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.687957+0000","last_change":"2024-09-12T09:31:21.272112+0000","last_active":"2024-09-12T09:31:25.687957+0000","last_peered":"2024-09-12T09:31:25.687957+0000","last_clean":"2024-09-12T09:31:25.687957+0000","last_became_active":"2024-09-12T09:31:21.271582+0000","last_became_peered":"2024-09-12T09:31:21.271582+0000","last_unstale":"2024-09-12T09:31:25.687957+0000","last_undegraded":"2024-09-12T09:31:25.687957+0000","last_fullsized":"2024-09-12T09:31:25.687957+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:48.094875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'179","reported_seq":5962,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.703206+0000","last_change":"2024-09-12T09:31:12.230044+0000","last_active":"2024-09-12T09:31:25.703206+0000","last_peered":"2024-09-12T09:31:25.703206+0000","last_clean":"2024-09-12T09:31:25.703206+0000","last_became_active":"2024-09-12T09:31:12.228291+0000","last_became_peered":"2024-09-12T09:31:12.228291+0000","last_unstale":"2024-09-12T09:31:25.703206+0000","last_undegraded":"2024-09-12T09:31:25.703206+0000","last_fullsized":"2024-09-12T09:31:25.703206+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:33.053900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":35100889,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":6251,"num_read_kb":23948,"num_write":171,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":11634641,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.688443+0000","last_change":"2024-09-12T09:31:21.271701+0000","last_active":"2024-09-12T09:31:25.688443+0000","last_peered":"2024-09-12T09:31:25.688443+0000","last_clean":"2024-09-12T09:31:25.688443+0000","last_became_active":"2024-09-12T09:31:21.271367+0000","last_became_peered":"2024-09-12T09:31:21.271367+0000","last_unstale":"2024-09-12T09:31:25.688443+0000","last_undegraded":"2024-09-12T09:31:25.688443+0000","last_fullsized":"2024-09-12T09:31:25.688443+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:11.579282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"117'224","reported_seq":2271,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.741855+0000","last_change":"2024-09-12T09:30:33.673235+0000","last_active":"2024-09-12T09:31:26.741855+0000","last_peered":"2024-09-12T09:31:26.741855+0000","last_clean":"2024-09-12T09:31:26.741855+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:31:26.741855+0000","last_undegraded":"2024-09-12T09:31:26.741855+0000","last_fullsized":"2024-09-12T09:31:26.741855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:33.028634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":60066961,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":16,"num_read":1912,"num_read_kb":17505,"num_write":282,"num_write_kb":24225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711421+0000","last_change":"2024-09-12T09:31:09.176142+0000","last_active":"2024-09-12T09:31:25.711421+0000","last_peered":"2024-09-12T09:31:25.711421+0000","last_clean":"2024-09-12T09:31:25.711421+0000","last_became_active":"2024-09-12T09:31:09.175686+0000","last_became_peered":"2024-09-12T09:31:09.175686+0000","last_unstale":"2024-09-12T09:31:25.711421+0000","last_undegraded":"2024-09-12T09:31:25.711421+0000","last_fullsized":"2024-09-12T09:31:25.711421+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:50.409345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"116'807","reported_seq":6392,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.986911+0000","last_change":"2024-09-12T09:30:46.985105+0000","last_active":"2024-09-12T09:31:25.986911+0000","last_peered":"2024-09-12T09:31:25.986911+0000","last_clean":"2024-09-12T09:31:25.986911+0000","last_became_active":"2024-09-12T09:30:46.983676+0000","last_became_peered":"2024-09-12T09:30:46.983676+0000","last_unstale":"2024-09-12T09:31:25.986911+0000","last_undegraded":"2024-09-12T09:31:25.986911+0000","last_fullsized":"2024-09-12T09:31:25.986911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:42.764211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":37940621,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":15,"num_read":9280,"num_read_kb":27658,"num_write":1348,"num_write_kb":46490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":7432750,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711386+0000","last_change":"2024-09-12T09:31:09.169240+0000","last_active":"2024-09-12T09:31:25.711386+0000","last_peered":"2024-09-12T09:31:25.711386+0000","last_clean":"2024-09-12T09:31:25.711386+0000","last_became_active":"2024-09-12T09:31:09.168549+0000","last_became_peered":"2024-09-12T09:31:09.168549+0000","last_unstale":"2024-09-12T09:31:25.711386+0000","last_undegraded":"2024-09-12T09:31:25.711386+0000","last_fullsized":"2024-09-12T09:31:25.711386+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:21.787537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"106'178","reported_seq":13500,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.682801+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:31:25.682801+0000","last_peered":"2024-09-12T09:31:25.682801+0000","last_clean":"2024-09-12T09:31:25.682801+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:31:25.682801+0000","last_undegraded":"2024-09-12T09:31:25.682801+0000","last_fullsized":"2024-09-12T09:31:25.682801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:28.296558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":36511988,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":13392,"num_read_kb":56328,"num_write":157,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"113'362","reported_seq":4701,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.703365+0000","last_change":"2024-09-12T09:31:24.690964+0000","last_active":"2024-09-12T09:31:25.703365+0000","last_peered":"2024-09-12T09:31:25.703365+0000","last_clean":"2024-09-12T09:31:25.703365+0000","last_became_active":"2024-09-12T09:31:24.688693+0000","last_became_peered":"2024-09-12T09:31:24.688693+0000","last_unstale":"2024-09-12T09:31:25.703365+0000","last_undegraded":"2024-09-12T09:31:25.703365+0000","last_fullsized":"2024-09-12T09:31:25.703365+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:14.949937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":25886015,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":24,"num_read":5226,"num_read_kb":20435,"num_write":434,"num_write_kb":9854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":5481109,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":243,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.682410+0000","last_change":"2024-09-12T09:31:21.278099+0000","last_active":"2024-09-12T09:31:25.682410+0000","last_peered":"2024-09-12T09:31:25.682410+0000","last_clean":"2024-09-12T09:31:25.682410+0000","last_became_active":"2024-09-12T09:31:21.273009+0000","last_became_peered":"2024-09-12T09:31:21.273009+0000","last_unstale":"2024-09-12T09:31:25.682410+0000","last_undegraded":"2024-09-12T09:31:25.682410+0000","last_fullsized":"2024-09-12T09:31:25.682410+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:46.392471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"113'270","reported_seq":4429,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.753711+0000","last_change":"2024-09-12T09:31:24.679935+0000","last_active":"2024-09-12T09:31:25.753711+0000","last_peered":"2024-09-12T09:31:25.753711+0000","last_clean":"2024-09-12T09:31:25.753711+0000","last_became_active":"2024-09-12T09:31:24.679426+0000","last_became_peered":"2024-09-12T09:31:24.679426+0000","last_unstale":"2024-09-12T09:31:25.753711+0000","last_undegraded":"2024-09-12T09:31:25.753711+0000","last_fullsized":"2024-09-12T09:31:25.753711+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:57.996868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":47485853,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":23,"num_read":4116,"num_read_kb":37024,"num_write":242,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":16260412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":212,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.683002+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:31:25.683002+0000","last_peered":"2024-09-12T09:31:25.683002+0000","last_clean":"2024-09-12T09:31:25.683002+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:31:25.683002+0000","last_undegraded":"2024-09-12T09:31:25.683002+0000","last_fullsized":"2024-09-12T09:31:25.683002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"117'268","reported_seq":12026,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.740850+0000","last_change":"2024-09-12T09:31:12.224528+0000","last_active":"2024-09-12T09:31:26.740850+0000","last_peered":"2024-09-12T09:31:26.740850+0000","last_clean":"2024-09-12T09:31:26.740850+0000","last_became_active":"2024-09-12T09:31:12.223598+0000","last_became_peered":"2024-09-12T09:31:12.223598+0000","last_unstale":"2024-09-12T09:31:26.740850+0000","last_undegraded":"2024-09-12T09:31:26.740850+0000","last_fullsized":"2024-09-12T09:31:26.740850+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:23:49.857645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":43332655,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":12523,"num_read_kb":36390,"num_write":253,"num_write_kb":23963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":10010383,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":117,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711139+0000","last_change":"2024-09-12T09:31:09.178478+0000","last_active":"2024-09-12T09:31:25.711139+0000","last_peered":"2024-09-12T09:31:25.711139+0000","last_clean":"2024-09-12T09:31:25.711139+0000","last_became_active":"2024-09-12T09:31:09.177755+0000","last_became_peered":"2024-09-12T09:31:09.177755+0000","last_unstale":"2024-09-12T09:31:25.711139+0000","last_undegraded":"2024-09-12T09:31:25.711139+0000","last_fullsized":"2024-09-12T09:31:25.711139+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:59:15.166490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"110'205","reported_seq":4874,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.707069+0000","last_change":"2024-09-12T09:31:24.690362+0000","last_active":"2024-09-12T09:31:25.707069+0000","last_peered":"2024-09-12T09:31:25.707069+0000","last_clean":"2024-09-12T09:31:25.707069+0000","last_became_active":"2024-09-12T09:31:24.688009+0000","last_became_peered":"2024-09-12T09:31:24.688009+0000","last_unstale":"2024-09-12T09:31:25.707069+0000","last_undegraded":"2024-09-12T09:31:25.707069+0000","last_fullsized":"2024-09-12T09:31:25.707069+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:46.419616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":50988415,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":18,"num_read":5672,"num_read_kb":44016,"num_write":217,"num_write_kb":11913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4653123,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.682477+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:31:25.682477+0000","last_peered":"2024-09-12T09:31:25.682477+0000","last_clean":"2024-09-12T09:31:25.682477+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:31:25.682477+0000","last_undegraded":"2024-09-12T09:31:25.682477+0000","last_fullsized":"2024-09-12T09:31:25.682477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":135,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711081+0000","last_change":"2024-09-12T09:31:21.283339+0000","last_active":"2024-09-12T09:31:25.711081+0000","last_peered":"2024-09-12T09:31:25.711081+0000","last_clean":"2024-09-12T09:31:25.711081+0000","last_became_active":"2024-09-12T09:31:21.277809+0000","last_became_peered":"2024-09-12T09:31:21.277809+0000","last_unstale":"2024-09-12T09:31:25.711081+0000","last_undegraded":"2024-09-12T09:31:25.711081+0000","last_fullsized":"2024-09-12T09:31:25.711081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:28.759873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"117'356","reported_seq":4447,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.903962+0000","last_change":"2024-09-12T09:30:46.981798+0000","last_active":"2024-09-12T09:31:25.903962+0000","last_peered":"2024-09-12T09:31:25.903962+0000","last_clean":"2024-09-12T09:31:25.903962+0000","last_became_active":"2024-09-12T09:30:46.980108+0000","last_became_peered":"2024-09-12T09:30:46.980108+0000","last_unstale":"2024-09-12T09:31:25.903962+0000","last_undegraded":"2024-09-12T09:31:25.903962+0000","last_fullsized":"2024-09-12T09:31:25.903962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:27:15.737511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":56763717,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":26,"num_read":6186,"num_read_kb":52852,"num_write":400,"num_write_kb":22937,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":10746792,"num_keys_recovered":21,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"113'411","reported_seq":3461,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.702960+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:31:25.702960+0000","last_peered":"2024-09-12T09:31:25.702960+0000","last_clean":"2024-09-12T09:31:25.702960+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:31:25.702960+0000","last_undegraded":"2024-09-12T09:31:25.702960+0000","last_fullsized":"2024-09-12T09:31:25.702960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:36:08.256907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":41228585,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":3814,"num_read_kb":24061,"num_write":629,"num_write_kb":12922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"116'262","reported_seq":5064,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.688189+0000","last_change":"2024-09-12T09:31:23.687201+0000","last_active":"2024-09-12T09:31:25.688189+0000","last_peered":"2024-09-12T09:31:25.688189+0000","last_clean":"2024-09-12T09:31:25.688189+0000","last_became_active":"2024-09-12T09:31:23.683236+0000","last_became_peered":"2024-09-12T09:31:23.683236+0000","last_unstale":"2024-09-12T09:31:25.688189+0000","last_undegraded":"2024-09-12T09:31:25.688189+0000","last_fullsized":"2024-09-12T09:31:25.688189+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:41:21.134542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":30567886,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":4796,"num_read_kb":13221,"num_write":342,"num_write_kb":7733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"117'341","reported_seq":5944,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.889066+0000","last_change":"2024-09-12T09:31:12.227245+0000","last_active":"2024-09-12T09:31:25.889066+0000","last_peered":"2024-09-12T09:31:25.889066+0000","last_clean":"2024-09-12T09:31:25.889066+0000","last_became_active":"2024-09-12T09:31:12.226373+0000","last_became_peered":"2024-09-12T09:31:12.226373+0000","last_unstale":"2024-09-12T09:31:25.889066+0000","last_undegraded":"2024-09-12T09:31:25.889066+0000","last_fullsized":"2024-09-12T09:31:25.889066+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:44.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":74170437,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":16,"num_read":6139,"num_read_kb":71505,"num_write":521,"num_write_kb":32209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"117'220","reported_seq":2442,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.055174+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:31:26.055174+0000","last_peered":"2024-09-12T09:31:26.055174+0000","last_clean":"2024-09-12T09:31:26.055174+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:31:26.055174+0000","last_undegraded":"2024-09-12T09:31:26.055174+0000","last_fullsized":"2024-09-12T09:31:26.055174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:27:48.410648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":37874329,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":3331,"num_read_kb":33061,"num_write":239,"num_write_kb":17185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"117'304","reported_seq":8018,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.593301+0000","last_change":"2024-09-12T09:30:33.675916+0000","last_active":"2024-09-12T09:31:26.593301+0000","last_peered":"2024-09-12T09:31:26.593301+0000","last_clean":"2024-09-12T09:31:26.593301+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:31:26.593301+0000","last_undegraded":"2024-09-12T09:31:26.593301+0000","last_fullsized":"2024-09-12T09:31:26.593301+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:43.004278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":76332188,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":8591,"num_read_kb":26651,"num_write":377,"num_write_kb":5287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"116'263","reported_seq":4910,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711629+0000","last_change":"2024-09-12T09:31:12.226125+0000","last_active":"2024-09-12T09:31:25.711629+0000","last_peered":"2024-09-12T09:31:25.711629+0000","last_clean":"2024-09-12T09:31:25.711629+0000","last_became_active":"2024-09-12T09:31:12.225225+0000","last_became_peered":"2024-09-12T09:31:12.225225+0000","last_unstale":"2024-09-12T09:31:25.711629+0000","last_undegraded":"2024-09-12T09:31:25.711629+0000","last_fullsized":"2024-09-12T09:31:25.711629+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:56.513836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":64933855,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":18,"num_read":5353,"num_read_kb":91991,"num_write":300,"num_write_kb":45016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":87332658,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"117'439","reported_seq":6195,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.869668+0000","last_change":"2024-09-12T09:31:12.222387+0000","last_active":"2024-09-12T09:31:26.869668+0000","last_peered":"2024-09-12T09:31:26.869668+0000","last_clean":"2024-09-12T09:31:26.869668+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:31:26.869668+0000","last_undegraded":"2024-09-12T09:31:26.869668+0000","last_fullsized":"2024-09-12T09:31:26.869668+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.506380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":96534306,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":22,"num_read":6499,"num_read_kb":76835,"num_write":561,"num_write_kb":44287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"109'367","reported_seq":2738,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.687900+0000","last_change":"2024-09-12T09:31:24.691099+0000","last_active":"2024-09-12T09:31:25.687900+0000","last_peered":"2024-09-12T09:31:25.687900+0000","last_clean":"2024-09-12T09:31:25.687900+0000","last_became_active":"2024-09-12T09:31:24.688190+0000","last_became_peered":"2024-09-12T09:31:24.688190+0000","last_unstale":"2024-09-12T09:31:25.687900+0000","last_undegraded":"2024-09-12T09:31:25.687900+0000","last_fullsized":"2024-09-12T09:31:25.687900+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:05.768915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":49465790,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":2704,"num_read_kb":21641,"num_write":443,"num_write_kb":24260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":17419997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"110'481","reported_seq":2847,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.711134+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:31:25.711134+0000","last_peered":"2024-09-12T09:31:25.711134+0000","last_clean":"2024-09-12T09:31:25.711134+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:31:25.711134+0000","last_undegraded":"2024-09-12T09:31:25.711134+0000","last_fullsized":"2024-09-12T09:31:25.711134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:16:09.051291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":64835995,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":19,"num_read":5296,"num_read_kb":16742,"num_write":636,"num_write_kb":5918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"116'299","reported_seq":11936,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:25.797772+0000","last_change":"2024-09-12T09:31:12.224349+0000","last_active":"2024-09-12T09:31:25.797772+0000","last_peered":"2024-09-12T09:31:25.797772+0000","last_clean":"2024-09-12T09:31:25.797772+0000","last_became_active":"2024-09-12T09:31:12.220300+0000","last_became_peered":"2024-09-12T09:31:12.220300+0000","last_unstale":"2024-09-12T09:31:25.797772+0000","last_undegraded":"2024-09-12T09:31:25.797772+0000","last_fullsized":"2024-09-12T09:31:25.797772+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:17:10.279934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":87999788,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":12243,"num_read_kb":104054,"num_write":398,"num_write_kb":55664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":132080671,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"117'159","reported_seq":1344,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-12T09:31:26.401233+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:31:26.401233+0000","last_peered":"2024-09-12T09:31:26.401233+0000","last_clean":"2024-09-12T09:31:26.401233+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:31:26.401233+0000","last_undegraded":"2024-09-12T09:31:26.401233+0000","last_fullsized":"2024-09-12T09:31:26.401233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:39:36.664276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":37599923,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":1506,"num_read_kb":9193,"num_write":245,"num_write_kb":11834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1674658407,"num_objects":1650,"num_object_clones":424,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":593,"num_read":208097,"num_read_kb":1373228,"num_write":14204,"num_write_kb":763910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2544,"num_bytes_recovered":614104834,"num_keys_recovered":482,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101580800,"data_stored":1357084228,"data_compressed":18921691,"data_compressed_allocated":89485312,"data_compressed_original":1349276326,"omap_allocated":0,"internal_metadata":0},"log_size":11017,"ondisk_log_size":11017,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48888,"kb_used_data":21864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94322952,"statfs":{"total":96636764160,"available":96586702848,"internally_reserved":0,"allocated":22388736,"data_stored":274336317,"data_compressed":4287564,"data_compressed_allocated":18526208,"data_compressed_original":271987371,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67576,"kb_used_data":18272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94304264,"statfs":{"total":96636764160,"available":96567566336,"internally_reserved":0,"allocated":18710528,"data_stored":213995761,"data_compressed":3833967,"data_compressed_allocated":15052800,"data_compressed_original":211866763,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5722112,"data_stored":37767457,"data_compressed":1182793,"data_compressed_allocated":3518464,"data_compressed_original":36659200,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59012,"kb_used_data":13932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94312828,"statfs":{"total":96636764160,"available":96576335872,"internally_reserved":0,"allocated":14266368,"data_stored":181809310,"data_compressed":2114784,"data_compressed_allocated":11411456,"data_compressed_original":180236856,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48980,"kb_used_data":17788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322860,"statfs":{"total":96636764160,"available":96586608640,"internally_reserved":0,"allocated":18214912,"data_stored":236908850,"data_compressed":3106128,"data_compressed_allocated":15282176,"data_compressed_original":235370723,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42336,"kb_used_data":11144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329504,"statfs":{"total":96636764160,"available":96593412096,"internally_reserved":0,"allocated":11411456,"data_stored":138651185,"data_compressed":1950933,"data_compressed_allocated":8654848,"data_compressed_original":137100256,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":60129542173,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542174,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16519168,"data_stored":251472716,"data_compressed":2140622,"data_compressed_allocated":14729216,"data_compressed_original":250205774,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3252224,"data_stored":33159688,"data_compressed":430613,"data_compressed_allocated":2797568,"data_compressed_original":32897110,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9363456,"data_stored":131730775,"data_compressed":1753335,"data_compressed_allocated":8138752,"data_compressed_original":130916785,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16912384,"data_stored":236466530,"data_compressed":3151493,"data_compressed_allocated":15335424,"data_compressed_original":235563235,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17321984,"data_stored":213341380,"data_compressed":3833967,"data_compressed_allocated":15052800,"data_compressed_original":211866763,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12935168,"data_stored":181226356,"data_compressed":2153092,"data_compressed_allocated":11452416,"data_compressed_original":180298296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4288512,"data_stored":36522689,"data_compressed":1176139,"data_compressed_allocated":3481600,"data_compressed_original":36069376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20987904,"data_stored":273164094,"data_compressed":4282430,"data_compressed_allocated":18497536,"data_compressed_original":271458987,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:31:31.409 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [1, 4, 5, 2] 2024-09-12T09:31:31.409 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '1', '4', '5', '2'] 2024-09-12T09:31:31.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 1 4 5 2 2024-09-12T09:31:31.410 INFO:teuthology.orchestra.run.smithi007.stdout:4618 read 0x47a7da4 thru 0x47b481a (0xca77 bytes) 2024-09-12T09:31:31.410 INFO:teuthology.orchestra.run.smithi007.stdout:4619 writesame 0x955d8c thru 0x960047 data_size 0xa(0xa2bc bytes) 2024-09-12T09:31:31.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:31.436 INFO:teuthology.orchestra.run.smithi007.stdout:4622 write 0xd156b6d thru 0xd15f1c6 (0x865a bytes) 2024-09-12T09:31:31.437 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:31.435+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:4623 trunc from 0xd15f1c7 to 0xc600452 2024-09-12T09:31:31.464 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.464 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.473 INFO:teuthology.orchestra.run.smithi007.stdout:4626 writesame 0xd0f8c44 thru 0xd1015d7 data_size 0x1e(0x8994 bytes) 2024-09-12T09:31:31.473 INFO:teuthology.orchestra.run.smithi007.stdout:4627 write 0xa7a720b thru 0xa7a9b40 (0x2936 bytes) 2024-09-12T09:31:31.478 INFO:teuthology.orchestra.run.smithi007.stdout:4628 read 0x52b3e5 thru 0x5393ee (0xe00a bytes) 2024-09-12T09:31:31.479 INFO:teuthology.orchestra.run.smithi007.stdout:4629 write 0x91328d1 thru 0x913707d (0x47ad bytes) 2024-09-12T09:31:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.487 INFO:teuthology.orchestra.run.smithi007.stdout:4631 trunc from 0xd1015d8 to 0xc360726 2024-09-12T09:31:31.495 INFO:teuthology.orchestra.run.smithi007.stdout:4632 trunc from 0xc360726 to 0xb7add1d 2024-09-12T09:31:31.514 INFO:teuthology.orchestra.run.smithi007.stdout:4633 write 0xe55a46c thru 0xe5606aa (0x623f bytes) 2024-09-12T09:31:31.528 INFO:teuthology.orchestra.run.smithi007.stdout:4635 punch from 0x58a84ac to 0x58aa68f, (0x21e3 bytes) 2024-09-12T09:31:31.561 INFO:teuthology.orchestra.run.smithi007.stdout:4637 writesame 0x63b9838 thru 0x63bc9f4 data_size 0x11(0x31bd bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4639 punch from 0x6e90bf0 to 0x6e9f577, (0xe987 bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4640 write 0x9d3e229 thru 0x9d45669 (0x7441 bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4641 punch from 0xc2d06aa to 0xc2dff85, (0xf8db bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4643 read 0x15a6298 thru 0x15a78f6 (0x165f bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4644 punch from 0xd814ba7 to 0xd81567d, (0xad6 bytes) 2024-09-12T09:31:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4645 trunc from 0xe5606ab to 0x6bb52c8 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898625274, 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:31.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898575666, 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:31.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:31.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898541924, 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:31.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.580 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89849488599999, 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:31.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:31.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898437046, 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:31.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898377959, 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:31.590 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:31.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:31.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898343772, 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:31.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:31.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898312436, 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:31.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898282395, 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:31.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:31.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89825114999999, 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:31.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:31.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898209381, 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:31.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89817791500001, 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:31.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89813037499999, 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:31.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898097338, 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:31.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:31.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.898052318, 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:31.636 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89801966, 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:31.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:31.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.897987845, 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:31.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.897929932, 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:31.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:31.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:31.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89788795, 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.897851968, 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:31.666 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:31.666 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:31.559+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:31.667 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:31.668 INFO:teuthology.orchestra.run.smithi007.stdout:4646 write 0x5f3a5d thru 0x5f6bc8 (0x316c bytes) 2024-09-12T09:31:31.668 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:4647 read 0x172014c thru 0x1721795 (0x164a bytes) 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:4650 read 0x448e9e5 thru 0x449629e (0x78ba bytes) 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:4651 trunc from 0x6bb52c8 to 0x8775365 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:4653 read 0x109ba98 thru 0x10aa35f (0xe8c8 bytes) 2024-09-12T09:31:31.669 INFO:teuthology.orchestra.run.smithi007.stdout:4654 write 0x6f82636 thru 0x6f83d9e (0x1769 bytes) 2024-09-12T09:31:31.670 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:31.670 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:31.670 INFO:teuthology.orchestra.run.smithi007.stdout:4656 read 0xd1efce thru 0xd2da48 (0xea7b bytes) 2024-09-12T09:31:31.671 INFO:teuthology.orchestra.run.smithi007.stdout:4657 trunc from 0x8775365 to 0xb4ada02 2024-09-12T09:31:31.671 INFO:teuthology.orchestra.run.smithi007.stdout:4658 read 0x28e7d82 thru 0x28f4975 (0xcbf4 bytes) 2024-09-12T09:31:31.671 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:31.671 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:31.671 INFO:teuthology.orchestra.run.smithi007.stdout:4661 write 0xe272188 thru 0xe2797e3 (0x765c bytes) 2024-09-12T09:31:31.672 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:31.672 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:31.672 INFO:teuthology.orchestra.run.smithi007.stdout:4662 punch from 0x1f22251 to 0x1f2f2ee, (0xd09d bytes) 2024-09-12T09:31:31.672 INFO:teuthology.orchestra.run.smithi007.stdout:4663 trunc from 0xe2797e4 to 0x47629b6 2024-09-12T09:31:31.673 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:31.666+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:31.676 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:31.676 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:31.690 INFO:teuthology.orchestra.run.smithi007.stdout:4665 read 0x1065b52 thru 0x1067d57 (0x2206 bytes) 2024-09-12T09:31:31.696 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:31.697 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:31.704 INFO:teuthology.orchestra.run.smithi007.stdout:4666 writesame 0x811790d thru 0x811f422 data_size 0x2e(0x7b16 bytes) 2024-09-12T09:31:31.704 INFO:teuthology.orchestra.run.smithi007.stdout:4667 write 0x49496c4 thru 0x49543e9 (0xad26 bytes) 2024-09-12T09:31:31.719 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.719 INFO:teuthology.orchestra.run.smithi007.stdout:4670 punch from 0x5fd5aea to 0x5fe3a7f, (0xdf95 bytes) 2024-09-12T09:31:31.719 INFO:teuthology.orchestra.run.smithi007.stdout:4672 writesame 0x481f2c1 thru 0x4821e8d data_size 0x1(0x2bcd bytes) 2024-09-12T09:31:31.720 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:31.735 INFO:teuthology.orchestra.run.smithi007.stdout:4673 trunc from 0x811f423 to 0x3e439c1 2024-09-12T09:31:31.745 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:31.781 INFO:teuthology.orchestra.run.smithi007.stdout:4676 writesame 0x88d2cdb thru 0x88dfa8d data_size 0x3(0xcdb3 bytes) 2024-09-12T09:31:31.781 INFO:teuthology.orchestra.run.smithi007.stdout:4677 write 0x5107c28 thru 0x51080ee (0x4c7 bytes) 2024-09-12T09:31:31.781 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:31.779+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:31.789 INFO:teuthology.orchestra.run.smithi007.stdout:4678 punch from 0x9f0c33 to 0x9f71ee, (0x65bb bytes) 2024-09-12T09:31:31.789 INFO:teuthology.orchestra.run.smithi007.stdout:4679 punch from 0x103add9 to 0x103c3c7, (0x15ee bytes) 2024-09-12T09:31:31.789 INFO:teuthology.orchestra.run.smithi007.stdout:4680 write 0x59a6bf0 thru 0x59ac1fc (0x560d bytes) 2024-09-12T09:31:31.797 INFO:teuthology.orchestra.run.smithi007.stdout:4682 read 0x243cc08 thru 0x244a1d0 (0xd5c9 bytes) 2024-09-12T09:31:31.799 INFO:teuthology.orchestra.run.smithi007.stdout:4683 read 0x6eadfb3 thru 0x6ebb3c5 (0xd413 bytes) 2024-09-12T09:31:31.812 INFO:teuthology.orchestra.run.smithi007.stdout:4684 writesame 0xaefabeb thru 0xaefba92 data_size 0x1c(0xea8 bytes) 2024-09-12T09:31:31.812 INFO:teuthology.orchestra.run.smithi007.stdout:4685 write 0x9e5f925 thru 0x9e5ff9a (0x676 bytes) 2024-09-12T09:31:31.817 INFO:teuthology.orchestra.run.smithi007.stdout:4686 read 0x5a95f79 thru 0x5a97f5b (0x1fe3 bytes) 2024-09-12T09:31:31.817 INFO:teuthology.orchestra.run.smithi007.stdout:4687 write 0x5b10b26 thru 0x5b1360d (0x2ae8 bytes) 2024-09-12T09:31:31.821 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:31.821 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:31.823 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.823 INFO:teuthology.orchestra.run.smithi007.stdout:4692 read 0x697cde6 thru 0x6989905 (0xcb20 bytes) 2024-09-12T09:31:31.823 INFO:teuthology.orchestra.run.smithi007.stdout:4693 read 0x82a1a73 thru 0x82a61ca (0x4758 bytes) 2024-09-12T09:31:31.823 INFO:teuthology.orchestra.run.smithi007.stdout:4694 write 0x7e1df8d thru 0x7e1f59f (0x1613 bytes) 2024-09-12T09:31:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:4695 writesame 0x6c3da11 thru 0x6c4d290 data_size 0x8e(0xf880 bytes) 2024-09-12T09:31:31.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:4697 read 0x311b783 thru 0x3121506 (0x5d84 bytes) 2024-09-12T09:31:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:4698 write 0x5e2b2b thru 0x5ede94 (0xb36a bytes) 2024-09-12T09:31:31.869 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:31.867+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:31.876 INFO:teuthology.orchestra.run.smithi007.stdout:4699 write 0xdbfe398 thru 0xdc0bb3e (0xd7a7 bytes) 2024-09-12T09:31:31.890 INFO:teuthology.orchestra.run.smithi007.stdout:4700 read 0x887e385 thru 0x8889df2 (0xba6e bytes) 2024-09-12T09:31:31.890 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.890 INFO:teuthology.orchestra.run.smithi007.stdout:4703 read 0x3a774e8 thru 0x3a77529 (0x42 bytes) 2024-09-12T09:31:31.890 INFO:teuthology.orchestra.run.smithi007.stdout:4704 writesame 0x3619d25 thru 0x3620ce3 data_size 0x1(0x6fbf bytes) 2024-09-12T09:31:31.905 INFO:teuthology.orchestra.run.smithi007.stdout:4706 write 0x35d4aab thru 0x35dfb33 (0xb089 bytes) 2024-09-12T09:31:31.915 INFO:teuthology.orchestra.run.smithi007.stdout:4707 read 0x5cb1cd8 thru 0x5cb5cfb (0x4024 bytes) 2024-09-12T09:31:31.916 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:31.917 INFO:teuthology.orchestra.run.smithi007.stdout:4709 trunc from 0xdc0bb3f to 0xd0c9e15 2024-09-12T09:31:31.938 INFO:teuthology.orchestra.run.smithi007.stdout:4710 read 0x845d164 thru 0x84682f9 (0xb196 bytes) 2024-09-12T09:31:31.938 INFO:teuthology.orchestra.run.smithi007.stdout:4711 write 0xa7c5cb2 thru 0xa7cb1ff (0x554e bytes) 2024-09-12T09:31:31.950 INFO:teuthology.orchestra.run.smithi007.stdout:4712 writesame 0x155d46 thru 0x1656e6 data_size 0x1(0xf9a1 bytes) 2024-09-12T09:31:31.971 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:31.967+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:31.983 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:31.984 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.007 INFO:teuthology.orchestra.run.smithi007.stdout:4714 write 0x86badbf thru 0x86bbc03 (0xe45 bytes) 2024-09-12T09:31:32.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:4717 trunc from 0xd0c9e15 to 0xc3be7e 2024-09-12T09:31:32.072 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:32.070+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:32.135 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.135 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.137 INFO:teuthology.orchestra.run.smithi007.stdout:4718 write 0xe225503 thru 0xe229b0c (0x460a bytes) 2024-09-12T09:31:32.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:32.151 INFO:teuthology.orchestra.run.smithi007.stdout:4719 write 0x21a76a2 thru 0x21abc8a (0x45e9 bytes) 2024-09-12T09:31:32.171 INFO:teuthology.orchestra.run.smithi007.stdout:4721 write 0xe4a450d thru 0xe4a7ef7 (0x39eb bytes) 2024-09-12T09:31:32.179 INFO:teuthology.orchestra.run.smithi007.stdout:4723 clone 42 order 21 su 8192 sc 5 2024-09-12T09:31:32.192 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:32.190+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:32.274 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:32.270+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:32.285 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.285 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:32.388 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:32.387+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:32.465 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.465 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.476 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:32.475+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:32.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:32.577 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:32.574+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:32.648 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.648 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:32.708 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:32.707+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:32.717 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1e pg_upmap_items mapping to [1->4,5->2] 2024-09-12T09:31:32.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:31:32.778 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:32.774+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:32.805 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.806 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:32.879 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:32.875+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:32.893 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone41 from 0xe4a7ef8 (overlap 0x373f68) to 0x0 2024-09-12T09:31:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:32.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:33.012 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:33.011+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:33.081 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:33.079+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:33.140 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:33.140 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:33.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:33.174 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-12T09:31:33.183 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:33.178+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:33.187 INFO:teuthology.orchestra.run.smithi007.stdout:4724 write 0xd3c7207 thru 0xd3c8f83 (0x1d7d bytes) 2024-09-12T09:31:33.206 INFO:teuthology.orchestra.run.smithi007.stdout:4725 punch from 0xc89eac0 to 0xc8ad41e, (0xe95e bytes) 2024-09-12T09:31:33.206 INFO:teuthology.orchestra.run.smithi007.stdout:4726 read 0x55f27f3 thru 0x55f42ac (0x1aba bytes) 2024-09-12T09:31:33.206 INFO:teuthology.orchestra.run.smithi007.stdout:4727 write 0x76647ff thru 0x7672df9 (0xe5fb bytes) 2024-09-12T09:31:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:4732 write 0xc01d874 thru 0xc0244c5 (0x6c52 bytes) 2024-09-12T09:31:33.240 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.240 INFO:teuthology.orchestra.run.smithi007.stdout:4734 write 0x8970c3 thru 0x89c81f (0x575d bytes) 2024-09-12T09:31:33.268 INFO:teuthology.orchestra.run.smithi007.stdout:4737 writesame 0x9aa12bf thru 0x9aa2415 data_size 0x1(0x1157 bytes) 2024-09-12T09:31:33.279 INFO:teuthology.orchestra.run.smithi007.stdout:4738 write 0x6537df0 thru 0x653ace9 (0x2efa bytes) 2024-09-12T09:31:33.283 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:33.278+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:33.290 INFO:teuthology.orchestra.run.smithi007.stdout:4739 write 0x94a277b thru 0x94aff24 (0xd7aa bytes) 2024-09-12T09:31:33.309 INFO:teuthology.orchestra.run.smithi007.stdout:4740 read 0x43a410f thru 0x43b17b5 (0xd6a7 bytes) 2024-09-12T09:31:33.309 INFO:teuthology.orchestra.run.smithi007.stdout:4742 read 0x2af1009 thru 0x2af9917 (0x890f bytes) 2024-09-12T09:31:33.309 INFO:teuthology.orchestra.run.smithi007.stdout:4743 read 0xbe77a99 thru 0xbe78968 (0xed0 bytes) 2024-09-12T09:31:33.310 INFO:teuthology.orchestra.run.smithi007.stdout:4744 writesame 0xa427d87 thru 0xa42a158 data_size 0xe(0x23d2 bytes) 2024-09-12T09:31:33.315 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:33.315 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:33.321 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.321 INFO:teuthology.orchestra.run.smithi007.stdout:4747 read 0x35f0797 thru 0x35f3a19 (0x3283 bytes) 2024-09-12T09:31:33.321 INFO:teuthology.orchestra.run.smithi007.stdout:4748 read 0x7656539 thru 0x7659824 (0x32ec bytes) 2024-09-12T09:31:33.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:33.352 INFO:teuthology.orchestra.run.smithi007.stdout:4749 trunc from 0xe4a7ef8 to 0x1120c9b 2024-09-12T09:31:33.396 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:33.395+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:33.430 INFO:teuthology.orchestra.run.smithi007.stdout:4752 write 0x6346f4d thru 0x634c204 (0x52b8 bytes) 2024-09-12T09:31:33.445 INFO:teuthology.orchestra.run.smithi007.stdout:4753 write 0x1f0000e thru 0x1f00769 (0x75c bytes) 2024-09-12T09:31:33.478 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:33.478 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:33.484 INFO:teuthology.orchestra.run.smithi007.stdout:4755 write 0xea546f1 thru 0xea58088 (0x3998 bytes) 2024-09-12T09:31:33.484 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:33.482+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:33.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:33.522 INFO:teuthology.orchestra.run.smithi007.stdout:4756 trunc from 0xea58089 to 0x260ac70 2024-09-12T09:31:33.547 INFO:teuthology.orchestra.run.smithi007.stdout:4757 write 0x343dac thru 0x34f86c (0xbac1 bytes) 2024-09-12T09:31:33.585 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:33.583+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:33.592 INFO:teuthology.orchestra.run.smithi007.stdout:4758 writesame 0xe9ebf5a thru 0xe9ed47e data_size 0x1(0x1525 bytes) 2024-09-12T09:31:33.592 INFO:teuthology.orchestra.run.smithi007.stdout:4762 write 0xe81871a thru 0xe81bf9b (0x3882 bytes) 2024-09-12T09:31:33.603 INFO:teuthology.orchestra.run.smithi007.stdout:4765 writesame 0xc0a5f70 thru 0xc0abc2d data_size 0x9(0x5cbe bytes) 2024-09-12T09:31:33.621 INFO:teuthology.orchestra.run.smithi007.stdout:4766 punch from 0xc3ca15 to 0xc3cf11, (0x4fc bytes) 2024-09-12T09:31:33.621 INFO:teuthology.orchestra.run.smithi007.stdout:4767 read 0x8a1ff08 thru 0x8a2bbca (0xbcc3 bytes) 2024-09-12T09:31:33.621 INFO:teuthology.orchestra.run.smithi007.stdout:4768 write 0xa69b1bf thru 0xa69ec4c (0x3a8e bytes) 2024-09-12T09:31:33.640 INFO:teuthology.orchestra.run.smithi007.stdout:4769 trunc from 0xe9ed47f to 0x999ceb7 2024-09-12T09:31:33.660 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:33.660 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:33.668 INFO:teuthology.orchestra.run.smithi007.stdout:4770 write 0x1cf9b62 thru 0x1cfc559 (0x29f8 bytes) 2024-09-12T09:31:33.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:33.695 INFO:teuthology.orchestra.run.smithi007.stdout:4771 writesame 0xb95d830 thru 0xb967831 data_size 0x3(0xa002 bytes) 2024-09-12T09:31:33.695 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:33.691+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:33.705 INFO:teuthology.orchestra.run.smithi007.stdout:4772 trunc from 0xb967832 to 0x59a04d4 2024-09-12T09:31:33.732 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:31:33.745 INFO:teuthology.orchestra.run.smithi007.stdout:4773 write 0xd7be9ce thru 0xd7c00a6 (0x16d9 bytes) 2024-09-12T09:31:33.757 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.757 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.758 INFO:teuthology.orchestra.run.smithi007.stdout:4776 writesame 0x8833f81 thru 0x8841e65 data_size 0x1(0xdee5 bytes) 2024-09-12T09:31:33.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:33.788 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:33.782+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:33.796 INFO:teuthology.orchestra.run.smithi007.stdout:4777 read 0x2e53239 thru 0x2e5ba70 (0x8838 bytes) 2024-09-12T09:31:33.796 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.796 INFO:teuthology.orchestra.run.smithi007.stdout:4780 writesame 0xc10e341 thru 0xc111564 data_size 0x2(0x3224 bytes) 2024-09-12T09:31:33.807 INFO:teuthology.orchestra.run.smithi007.stdout:4782 write 0xcce13fb thru 0xcce4013 (0x2c19 bytes) 2024-09-12T09:31:33.824 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:33.824 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:33.831 INFO:teuthology.orchestra.run.smithi007.stdout:4783 trunc from 0xd7c00a7 to 0x5418fea 2024-09-12T09:31:33.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:33.857 INFO:teuthology.orchestra.run.smithi007.stdout:4784 read 0x37091f3 thru 0x37169b9 (0xd7c7 bytes) 2024-09-12T09:31:33.872 INFO:teuthology.orchestra.run.smithi007.stdout:4785 write 0x99969eb thru 0x999e7a0 (0x7db6 bytes) 2024-09-12T09:31:33.888 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:33.886+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:33.893 INFO:teuthology.orchestra.run.smithi007.stdout:4786 punch from 0x4434eed to 0x44413d1, (0xc4e4 bytes) 2024-09-12T09:31:33.893 INFO:teuthology.orchestra.run.smithi007.stdout:4787 read 0x267a351 thru 0x26845c4 (0xa274 bytes) 2024-09-12T09:31:33.893 INFO:teuthology.orchestra.run.smithi007.stdout:4788 punch from 0x1b73c41 to 0x1b81cf7, (0xe0b6 bytes) 2024-09-12T09:31:33.893 INFO:teuthology.orchestra.run.smithi007.stdout:4790 write 0x8f5190f thru 0x8f51bed (0x2df bytes) 2024-09-12T09:31:33.899 INFO:teuthology.orchestra.run.smithi007.stdout:4791 writesame 0x9023c86 thru 0x90313af data_size 0x1(0xd72a bytes) 2024-09-12T09:31:33.921 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:33.921 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:33.926 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:33.926 INFO:teuthology.orchestra.run.smithi007.stdout:4793 read 0x2903fe5 thru 0x290ccee (0x8d0a bytes) 2024-09-12T09:31:33.926 INFO:teuthology.orchestra.run.smithi007.stdout:4794 punch from 0x551469a to 0x5524063, (0xf9c9 bytes) 2024-09-12T09:31:33.926 INFO:teuthology.orchestra.run.smithi007.stdout:4796 write 0x608567f thru 0x608d2bc (0x7c3e bytes) 2024-09-12T09:31:33.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:33.940 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:33.948 INFO:teuthology.orchestra.run.smithi007.stdout:4797 trunc from 0x999e7a1 to 0xe64b576 2024-09-12T09:31:33.956 INFO:teuthology.orchestra.run.smithi007.stdout:4799 read 0xd85827a thru 0xd85da8f (0x5816 bytes) 2024-09-12T09:31:33.956 INFO:teuthology.orchestra.run.smithi007.stdout:4801 write 0xc556633 thru 0xc55b1f2 (0x4bc0 bytes) 2024-09-12T09:31:33.958 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:33.958 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:33.977 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:33.985 INFO:teuthology.orchestra.run.smithi007.stdout:4802 trunc from 0xe64b576 to 0x1ae53bb 2024-09-12T09:31:33.986 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:33.986 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:33.986 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:33.986 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:33.987 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:33.989 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:33.986+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:34.001 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:34.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:34.017 INFO:teuthology.orchestra.run.smithi007.stdout:4803 trunc from 0x1ae53bb to 0x7d6d5c 2024-09-12T09:31:34.025 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:34.041 INFO:teuthology.orchestra.run.smithi007.stdout:4804 trunc from 0x7d6d5c to 0x13f7e1d 2024-09-12T09:31:34.047 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:34.047 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:34.049 INFO:teuthology.orchestra.run.smithi007.stdout:4805 read 0x136fb56 thru 0x137d4e8 (0xd993 bytes) 2024-09-12T09:31:34.049 INFO:teuthology.orchestra.run.smithi007.stdout:4806 read 0x130a727 thru 0x130acf4 (0x5ce bytes) 2024-09-12T09:31:34.050 INFO:teuthology.orchestra.run.smithi007.stdout:4810 punch from 0x4ff66a to 0x50e851, (0xf1e7 bytes) 2024-09-12T09:31:34.050 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.069 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:34.075 INFO:teuthology.orchestra.run.smithi007.stdout:4812 writesame 0xc804932 thru 0xc810f08 data_size 0x1(0xc5d7 bytes) 2024-09-12T09:31:34.076 INFO:teuthology.orchestra.run.smithi007.stdout:4814 write 0x8fe6dc7 thru 0x8ff3068 (0xc2a2 bytes) 2024-09-12T09:31:34.089 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:34.086+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:34.095 INFO:teuthology.orchestra.run.smithi007.stdout:4816 writesame 0x148b1fa thru 0x149b163 data_size 0x1(0xff6a bytes) 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.706022335, 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:31:34.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.705997103999998, 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:34.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:31:34.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.705959184000001, 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:31:34.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:31:34.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.705929183000002, 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:31:34.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.697894761999997, 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:31:34.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:31:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.697573888999997, 2024-09-12T09:31:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:31:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149100 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 865280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.697150063000002, 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097623149000000006, 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149100 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444557+0000", 2024-09-12T09:31:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444558+0000", 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0300000000000001e-06 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444563+0000", 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6359999999999999e-06 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444578+0000", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5073e-05 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444587+0000", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3850000000000002e-06 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541869+0000", 2024-09-12T09:31:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097281934 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541948+0000", 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9725e-05 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542180+0000", 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023136599999999999 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.697098584000003, 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:31:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.697069835999997, 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:31:34.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:31:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.696519369999997, 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:31:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.696476548, 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:31:34.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:31:34.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:34.196 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.696430657000001, 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:31:34.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.695919099000001, 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:31:34.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.695871320000002, 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:31:34.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.695815674000002, 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.695783652999999, 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:31:34.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.695219504000001, 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:31:34.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.694825602999998, 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:31:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.694433904999997, 2024-09-12T09:31:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:31:34.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3265677870000001, 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:34.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:34.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:34.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:31:34.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:34.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:4818 trunc from 0xc810f09 to 0x66e8418 2024-09-12T09:31:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:4819 write 0x48da0b7 thru 0x48e3072 (0x8fbc bytes) 2024-09-12T09:31:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:4820 punch from 0x3627458 to 0x362bb78, (0x4720 bytes) 2024-09-12T09:31:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:4822 write 0xb1095cf thru 0xb10ff8f (0x69c1 bytes) 2024-09-12T09:31:34.242 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:34.238+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:4823 writesame 0x26aff34 thru 0x26bf728 data_size 0x9(0xf7f5 bytes) 2024-09-12T09:31:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:4826 read 0x277db65 thru 0x277fc16 (0x20b2 bytes) 2024-09-12T09:31:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:4828 read 0x9db13f8 thru 0x9db3aa9 (0x26b2 bytes) 2024-09-12T09:31:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:4829 writesame 0x85bb856 thru 0x85c950b data_size 0x56(0xdcb6 bytes) 2024-09-12T09:31:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:4833 writesame 0xa83a25c thru 0xa83cfc8 data_size 0x1(0x2d6d bytes) 2024-09-12T09:31:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:4834 read 0x8b84850 thru 0x8b8b4be (0x6c6f bytes) 2024-09-12T09:31:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:4835 trunc from 0xb10ff90 to 0xd991dba 2024-09-12T09:31:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:4837 punch from 0xcef081e to 0xcef2990, (0x2172 bytes) 2024-09-12T09:31:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:4839 read 0x6877e19 thru 0x6878bf5 (0xddd bytes) 2024-09-12T09:31:34.341 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:34.338+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:4840 trunc from 0xd991dba to 0x625ae9c 2024-09-12T09:31:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:4842 punch from 0x187ab33 to 0x187d949, (0x2e16 bytes) 2024-09-12T09:31:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:4843 read 0x33051d1 thru 0x3308548 (0x3378 bytes) 2024-09-12T09:31:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:4846 write 0x19314d thru 0x1a2213 (0xf0c7 bytes) 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:183603 3.14 3:2bccdd27:::rbd_data.1191ec363dbc.0000000000000002:head [stat out=16b,write 327680~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:34.378240+0000", 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0041969260000000001, 2024-09-12T09:31:34.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042426979999999996, 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 183603 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378240+0000", 2024-09-12T09:31:34.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378240+0000", 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378242+0000", 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378259+0000", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7889000000000001e-05 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378275+0000", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5917999999999999e-05 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378284+0000", 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0180000000000007e-06 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378396+0000", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011131 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378584+0000", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000188223 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.378964+0000", 2024-09-12T09:31:34.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038029999999999997 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.381711+0000", 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027471420000000002 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:34.382392+0000", 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068082600000000004 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.388 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:31:34.388 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:31:34.388 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:34.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:4848 writesame 0xba84e03 thru 0xba90b19 data_size 0x1(0xbd17 bytes) 2024-09-12T09:31:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:4849 trunc from 0xba90b1a to 0xb9bbbec 2024-09-12T09:31:34.424 INFO:teuthology.orchestra.run.smithi007.stdout:4850 writesame 0xa3e002d thru 0xa3e2470 data_size 0x4(0x2444 bytes) 2024-09-12T09:31:34.444 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:34.442+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:34.454 INFO:teuthology.orchestra.run.smithi007.stdout:4851 write 0xdeb6077 thru 0xdebd796 (0x7720 bytes) 2024-09-12T09:31:34.499 INFO:teuthology.orchestra.run.smithi007.stdout:4853 trunc from 0xdebd797 to 0x1d843f5 2024-09-12T09:31:34.530 INFO:teuthology.orchestra.run.smithi007.stdout:4854 writesame 0x10e1ef0 thru 0x10e369a data_size 0x1(0x17ab bytes) 2024-09-12T09:31:34.543 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:34.539+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:34.573 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:34.573 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:34.583 INFO:teuthology.orchestra.run.smithi007.stdout:4855 writesame 0xb2109ff thru 0xb21f755 data_size 0x81(0xed57 bytes) 2024-09-12T09:31:34.584 INFO:teuthology.orchestra.run.smithi007.stdout:4856 write 0x14daf02 thru 0x14df6a5 (0x47a4 bytes) 2024-09-12T09:31:34.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:34.588 INFO:teuthology.orchestra.run.smithi007.stdout:4857 read 0x5e677bc thru 0x5e771a8 (0xf9ed bytes) 2024-09-12T09:31:34.588 INFO:teuthology.orchestra.run.smithi007.stdout:4858 write 0x7b571b3 thru 0x7b609b6 (0x9804 bytes) 2024-09-12T09:31:34.608 INFO:teuthology.orchestra.run.smithi007.stdout:4859 punch from 0x30e7db6 to 0x30ecea4, (0x50ee bytes) 2024-09-12T09:31:34.629 INFO:teuthology.orchestra.run.smithi007.stdout:4861 writesame 0xde0fe95 thru 0xde183eb data_size 0x1(0x8557 bytes) 2024-09-12T09:31:34.629 INFO:teuthology.orchestra.run.smithi007.stdout:4862 write 0x7619a3d thru 0x761af33 (0x14f7 bytes) 2024-09-12T09:31:34.638 INFO:teuthology.orchestra.run.smithi007.stdout:4863 write 0x3f2a549 thru 0x3f2d00b (0x2ac3 bytes) 2024-09-12T09:31:34.643 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:34.638+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:34.646 INFO:teuthology.orchestra.run.smithi007.stdout:4865 write 0x227a3da thru 0x2289130 (0xed57 bytes) 2024-09-12T09:31:34.660 INFO:teuthology.orchestra.run.smithi007.stdout:4867 read 0x9132117 thru 0x91401ee (0xe0d8 bytes) 2024-09-12T09:31:34.660 INFO:teuthology.orchestra.run.smithi007.stdout:4868 punch from 0x585e64b to 0x5867745, (0x90fa bytes) 2024-09-12T09:31:34.661 INFO:teuthology.orchestra.run.smithi007.stdout:4869 read 0x6592936 thru 0x659a4d5 (0x7ba0 bytes) 2024-09-12T09:31:34.661 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.661 INFO:teuthology.orchestra.run.smithi007.stdout:4873 read 0xcb03bfc thru 0xcb123fa (0xe7ff bytes) 2024-09-12T09:31:34.661 INFO:teuthology.orchestra.run.smithi007.stdout:4874 writesame 0xbddbbd4 thru 0xbde8277 data_size 0x2(0xc6a4 bytes) 2024-09-12T09:31:34.686 INFO:teuthology.orchestra.run.smithi007.stdout:4875 punch from 0x5e0e26a to 0x5e1e07f, (0xfe15 bytes) 2024-09-12T09:31:34.711 INFO:teuthology.orchestra.run.smithi007.stdout:4876 trunc from 0xde183ec to 0x4bdaa12 2024-09-12T09:31:34.723 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:34.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.432568083000007, 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:34.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:34.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.973071853999997, 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:34.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:34.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.599654903000001, 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:34.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:34.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.599607656000003, 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:34.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.599314579999998, 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.598752648000001, 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.598701285000004, 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:34.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:34.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:34.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.598128404999997, 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:34.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:34.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:34.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.597976106000004, 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:34.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.597619015999996, 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:34.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.596994046000006, 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:34.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.596942709999993, 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:34.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.596147920000007, 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:34.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.596073029999999, 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:34.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.596020710000005, 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:34.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.595972111999998, 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:34.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:34.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.595717647000001, 2024-09-12T09:31:34.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:34.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.592263356999993, 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:34.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 71.534694810000005, 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:34.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:34.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.063423573000001, 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:34.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:34.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:34.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:34.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:34.823 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:34.825 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:34.825 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:34.822+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4878 read 0x309409a thru 0x30a1543 (0xd4aa bytes) 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4879 writesame 0xb8489bb thru 0xb84c190 data_size 0x7(0x37d6 bytes) 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4880 writesame 0x3ee8bc2 thru 0x3eefc9a data_size 0x1(0x70d9 bytes) 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4881 trunc from 0xb84c191 to 0xabc95ec 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4882 punch from 0x11aa1e2 to 0x11ada4a, (0x3868 bytes) 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4884 read 0x2b79052 thru 0x2b7ab97 (0x1b46 bytes) 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:34.826 INFO:teuthology.orchestra.run.smithi007.stdout:4887 clone 43 order 20 su 262144 sc 8 2024-09-12T09:31:34.925 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:34.923+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:34.996 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:34.996 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:34.996 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:34.997 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:35.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:35.056 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:35.054+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:35.127 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:35.122+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:35.132 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:35.132 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:35.132 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:35.132 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:35.132 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:35.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:35.232 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:35.230+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:35.313 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.901485085, 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:35.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.901389823, 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:35.319 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:35.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:35.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.897705199, 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:35.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.895900109, 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:35.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.895353434, 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:35.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:35.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.894584991, 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:35.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.894174623, 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:35.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.89403544699999, 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.886256636, 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:35.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:35.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:35.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.883892622, 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:35.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.873140729, 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:35.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:35.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.872944338, 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.871449918, 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:35.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.171207057000004, 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:35.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:35.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.171184639000003, 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:35.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.171161498000004, 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:35.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:35.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.171136539000003, 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:35.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:35.402 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.127785024999994, 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:35.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:35.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 71.511306982999997, 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:35.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 71.511207693000003, 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:35.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:35.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.419 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:35.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:35.421 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:35.419+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:35.521 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:35.518+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:35.591 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:35.591 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:35.591 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:35.591 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:35.605 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:35.622 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:35.619+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:35.723 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:35.719+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:35.756 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:31:35.776 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:35.776 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:35.791 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:35.836 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:35.834+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:35.919 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:35.919 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:35.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:35.919 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:35.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279712606, 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:35.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:35.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27966299800001, 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:35.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:35.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.929 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27962925600001, 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:35.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279582218, 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279524378, 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27946529099999, 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279431104, 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279399768, 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279369727, 2024-09-12T09:31:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279338482, 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27929671299999, 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279265247, 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279217707, 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27918467000001, 2024-09-12T09:31:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.27913965, 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279106992, 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279075177, 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.279017264, 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.278975282, 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.2789393, 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:36.015 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:36.016 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone42 intact 2024-09-12T09:31:36.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:36.017 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:36.014+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:36.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:36.136 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:36.135+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:36.179 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-12T09:31:36.182 INFO:teuthology.orchestra.run.smithi007.stdout:4888 punch from 0xa73542b to 0xa735e4a, (0xa1f bytes) 2024-09-12T09:31:36.189 INFO:teuthology.orchestra.run.smithi007.stdout:4889 flatten 2024-09-12T09:31:36.196 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:36.196 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:36.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:36.218 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:36.215+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:36.250 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:36.250 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:36.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:36.283 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.283 INFO:teuthology.orchestra.run.smithi007.stdout:4891 write 0x1bb21ba thru 0x1bb5b20 (0x3967 bytes) 2024-09-12T09:31:36.289 INFO:teuthology.orchestra.run.smithi007.stdout:4893 writesame 0xa7670f0 thru 0xa76cb4c data_size 0xb(0x5a5d bytes) 2024-09-12T09:31:36.292 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:36.292 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:36.304 INFO:teuthology.orchestra.run.smithi007.stdout:4894 trunc from 0xabc95ec to 0x3807ece 2024-09-12T09:31:36.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:36.309 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:36.319 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:36.314+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:36.327 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:36.335 INFO:teuthology.orchestra.run.smithi007.stdout:4896 write 0x22b3918 thru 0x22bc79e (0x8e87 bytes) 2024-09-12T09:31:36.338 INFO:teuthology.orchestra.run.smithi007.stdout:4897 trunc from 0x3807ece to 0x2fd53cc 2024-09-12T09:31:36.347 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:36.360 INFO:teuthology.orchestra.run.smithi007.stdout:4900 writesame 0x5a71128 thru 0x5a7c601 data_size 0x7(0xb4da bytes) 2024-09-12T09:31:36.360 INFO:teuthology.orchestra.run.smithi007.stdout:4901 read 0x501eeed thru 0x502cb74 (0xdc88 bytes) 2024-09-12T09:31:36.363 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:36.363 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:36.363 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:36.363 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:36.363 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:36.368 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:36.372 INFO:teuthology.orchestra.run.smithi007.stdout:4902 write 0x8e7ea9b thru 0x8e88270 (0x97d6 bytes) 2024-09-12T09:31:36.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:36.381 INFO:teuthology.orchestra.run.smithi007.stdout:4904 punch from 0x5367c98 to 0x5372b05, (0xae6d bytes) 2024-09-12T09:31:36.381 INFO:teuthology.orchestra.run.smithi007.stdout:4905 write 0x1a9d211 thru 0x1aac7a9 (0xf599 bytes) 2024-09-12T09:31:36.385 INFO:teuthology.orchestra.run.smithi007.stdout:4906 punch from 0x64748a6 to 0x647f533, (0xac8d bytes) 2024-09-12T09:31:36.385 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.385 INFO:teuthology.orchestra.run.smithi007.stdout:4909 read 0x14a749b thru 0x14aee0a (0x7970 bytes) 2024-09-12T09:31:36.386 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.386 INFO:teuthology.orchestra.run.smithi007.stdout:4912 write 0x13d063e thru 0x13de388 (0xdd4b bytes) 2024-09-12T09:31:36.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:36.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:36.390 INFO:teuthology.orchestra.run.smithi007.stdout:4913 read 0x548fe2e thru 0x549ab11 (0xace4 bytes) 2024-09-12T09:31:36.390 INFO:teuthology.orchestra.run.smithi007.stdout:4914 read 0x616df1f thru 0x6178c4a (0xad2c bytes) 2024-09-12T09:31:36.399 INFO:teuthology.orchestra.run.smithi007.stdout:4915 trunc from 0x8e88271 to 0xe9e2e55 2024-09-12T09:31:36.412 INFO:teuthology.orchestra.run.smithi007.stdout:4916 trunc from 0xe9e2e55 to 0xb6181d9 2024-09-12T09:31:36.420 INFO:teuthology.orchestra.run.smithi007.stdout:4918 read 0x6abeffc thru 0x6ac4916 (0x591b bytes) 2024-09-12T09:31:36.420 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.424 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:36.422+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:36.431 INFO:teuthology.orchestra.run.smithi007.stdout:4922 write 0xd9a49b0 thru 0xd9b3480 (0xead1 bytes) 2024-09-12T09:31:36.436 INFO:teuthology.orchestra.run.smithi007.stdout:4923 write 0x894a152 thru 0x8958440 (0xe2ef bytes) 2024-09-12T09:31:36.453 INFO:teuthology.orchestra.run.smithi007.stdout:4924 trunc from 0xd9b3481 to 0x6dae4a9 2024-09-12T09:31:36.474 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.484 INFO:teuthology.orchestra.run.smithi007.stdout:4926 trunc from 0x6dae4a9 to 0x1e00af7 2024-09-12T09:31:36.508 INFO:teuthology.orchestra.run.smithi007.stdout:4927 read 0xd78947 thru 0xd7a251 (0x190b bytes) 2024-09-12T09:31:36.521 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:36.518+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:36.547 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:36.547 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:36.551 INFO:teuthology.orchestra.run.smithi007.stdout:4928 write 0xcc10615 thru 0xcc12b0d (0x24f9 bytes) 2024-09-12T09:31:36.558 INFO:teuthology.orchestra.run.smithi007.stdout:4929 writesame 0x769eb67 thru 0x76ac43e data_size 0x36(0xd8d8 bytes) 2024-09-12T09:31:36.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:36.566 INFO:teuthology.orchestra.run.smithi007.stdout:4930 write 0x25fc0a6 thru 0x25fcdd6 (0xd31 bytes) 2024-09-12T09:31:36.571 INFO:teuthology.orchestra.run.smithi007.stdout:4932 read 0xa3512c9 thru 0xa35c81d (0xb555 bytes) 2024-09-12T09:31:36.577 INFO:teuthology.orchestra.run.smithi007.stdout:4934 trunc from 0xcc12b0e to 0xaffc353 2024-09-12T09:31:36.607 INFO:teuthology.orchestra.run.smithi007.stdout:4936 trunc from 0xaffc353 to 0x5c29590 2024-09-12T09:31:36.622 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:36.618+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:36.625 INFO:teuthology.orchestra.run.smithi007.stdout:4937 write 0x3e8e910 thru 0x3e92896 (0x3f87 bytes) 2024-09-12T09:31:36.632 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.632 INFO:teuthology.orchestra.run.smithi007.stdout:4943 trunc from 0x5c29590 to 0x5fb25ed 2024-09-12T09:31:36.643 INFO:teuthology.orchestra.run.smithi007.stdout:4944 read 0x290b958 thru 0x2912bcc (0x7275 bytes) 2024-09-12T09:31:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:4945 write 0x9fbdab6 thru 0x9fc0140 (0x268b bytes) 2024-09-12T09:31:36.670 INFO:teuthology.orchestra.run.smithi007.stdout:4946 read 0x8e956d9 thru 0x8e9d379 (0x7ca1 bytes) 2024-09-12T09:31:36.671 INFO:teuthology.orchestra.run.smithi007.stdout:4947 read 0x9f7c0a9 thru 0x9f8619d (0xa0f5 bytes) 2024-09-12T09:31:36.671 INFO:teuthology.orchestra.run.smithi007.stdout:4948 write 0x3d671ef thru 0x3d6cfee (0x5e00 bytes) 2024-09-12T09:31:36.678 INFO:teuthology.orchestra.run.smithi007.stdout:4949 writesame 0x8307a92 thru 0x8309d90 data_size 0x1f(0x22ff bytes) 2024-09-12T09:31:36.686 INFO:teuthology.orchestra.run.smithi007.stdout:4950 trunc from 0x9fc0141 to 0x9cfdd8d 2024-09-12T09:31:36.703 INFO:teuthology.orchestra.run.smithi007.stdout:4952 write 0x4785131 thru 0x478a600 (0x54d0 bytes) 2024-09-12T09:31:36.704 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:36.704 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:36.711 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.711 INFO:teuthology.orchestra.run.smithi007.stdout:4954 writesame 0x83e26ad thru 0x83e99b3 data_size 0x1(0x7307 bytes) 2024-09-12T09:31:36.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:31:36.739 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:36.735+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:36.755 INFO:teuthology.orchestra.run.smithi007.stdout:4955 write 0x295c583 thru 0x295d542 (0xfc0 bytes) 2024-09-12T09:31:36.766 INFO:teuthology.orchestra.run.smithi007.stdout:4956 write 0x477f92a thru 0x478710a (0x77e1 bytes) 2024-09-12T09:31:36.772 INFO:teuthology.orchestra.run.smithi007.stdout:4957 read 0x749cf0a thru 0x74a0afc (0x3bf3 bytes) 2024-09-12T09:31:36.772 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.772 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:36.772 INFO:teuthology.orchestra.run.smithi007.stdout:4963 write 0x576092e thru 0x57677f5 (0x6ec8 bytes) 2024-09-12T09:31:36.783 INFO:teuthology.orchestra.run.smithi007.stdout:4968 trunc from 0x9cfdd8d to 0xa810de9 2024-09-12T09:31:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:4969 writesame 0xbdc3a06 thru 0xbdd230e data_size 0xd(0xe909 bytes) 2024-09-12T09:31:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:4970 read 0x77d9b7b thru 0x77dd756 (0x3bdc bytes) 2024-09-12T09:31:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:4972 writesame 0x86291e2 thru 0x86375d8 data_size 0x3(0xe3f7 bytes) 2024-09-12T09:31:36.824 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:36.818+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:36.840 INFO:teuthology.orchestra.run.smithi007.stdout:4973 read 0x6e55f7d thru 0x6e65183 (0xf207 bytes) 2024-09-12T09:31:36.861 INFO:teuthology.orchestra.run.smithi007.stdout:4974 trunc from 0xbdd230f to 0x2b42106 2024-09-12T09:31:36.871 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:36.871 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:36.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:36.928 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:36.927+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:36.929 INFO:teuthology.orchestra.run.smithi007.stdout:4977 write 0xd3bba7b thru 0xd3c8dcd (0xd353 bytes) 2024-09-12T09:31:36.940 INFO:teuthology.orchestra.run.smithi007.stdout:4978 read 0xaab6d1e thru 0xaabafc9 (0x42ac bytes) 2024-09-12T09:31:36.954 INFO:teuthology.orchestra.run.smithi007.stdout:4979 writesame 0xe41075b thru 0xe412f55 data_size 0x5(0x27fb bytes) 2024-09-12T09:31:36.955 INFO:teuthology.orchestra.run.smithi007.stdout:4980 write 0xe274ba9 thru 0xe279adf (0x4f37 bytes) 2024-09-12T09:31:36.964 INFO:teuthology.orchestra.run.smithi007.stdout:4981 punch from 0x350ea17 to 0x3512e1e, (0x4407 bytes) 2024-09-12T09:31:36.965 INFO:teuthology.orchestra.run.smithi007.stdout:4984 punch from 0x78e17d4 to 0x78e8c9f, (0x74cb bytes) 2024-09-12T09:31:36.965 INFO:teuthology.orchestra.run.smithi007.stdout:4985 read 0xf443dc thru 0xf46997 (0x25bc bytes) 2024-09-12T09:31:36.989 INFO:teuthology.orchestra.run.smithi007.stdout:4986 trunc from 0xe412f56 to 0x4b742 2024-09-12T09:31:37.026 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:37.023+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:37.062 INFO:teuthology.orchestra.run.smithi007.stdout:4987 trunc from 0x4b742 to 0x8ef5635 2024-09-12T09:31:37.063 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.063 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.078 INFO:teuthology.orchestra.run.smithi007.stdout:4988 read 0x7c6803 thru 0x7c8f67 (0x2765 bytes) 2024-09-12T09:31:37.078 INFO:teuthology.orchestra.run.smithi007.stdout:4991 punch from 0x37ec9cb to 0x37f8917, (0xbf4c bytes) 2024-09-12T09:31:37.078 INFO:teuthology.orchestra.run.smithi007.stdout:4992 write 0x3900207 thru 0x3902983 (0x277d bytes) 2024-09-12T09:31:37.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:37.085 INFO:teuthology.orchestra.run.smithi007.stdout:4993 read 0x4d498a6 thru 0x4d4c94e (0x30a9 bytes) 2024-09-12T09:31:37.086 INFO:teuthology.orchestra.run.smithi007.stdout:4994 punch from 0x55122a5 to 0x551a51a, (0x8275 bytes) 2024-09-12T09:31:37.086 INFO:teuthology.orchestra.run.smithi007.stdout:4995 write 0x4c133a5 thru 0x4c1680c (0x3468 bytes) 2024-09-12T09:31:37.100 INFO:teuthology.orchestra.run.smithi007.stdout:4997 trunc from 0x8ef5635 to 0x447d71a 2024-09-12T09:31:37.127 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:37.122+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:37.153 INFO:teuthology.orchestra.run.smithi007.stdout:4998 write 0xdd4fb6d thru 0xdd5525d (0x56f1 bytes) 2024-09-12T09:31:37.162 INFO:teuthology.orchestra.run.smithi007.stdout:4999 read 0xbec2d45 thru 0xbecb505 (0x87c1 bytes) 2024-09-12T09:31:37.162 INFO:teuthology.orchestra.run.smithi007.stdout:5000 write 0x8ff6ae8 thru 0x8ff7bd5 (0x10ee bytes) 2024-09-12T09:31:37.169 INFO:teuthology.orchestra.run.smithi007.stdout:5001 read 0x9b9df5f thru 0x9b9e41c (0x4be bytes) 2024-09-12T09:31:37.170 INFO:teuthology.orchestra.run.smithi007.stdout:5002 writesame 0x65f8a11 thru 0x65f92e3 data_size 0x1(0x8d3 bytes) 2024-09-12T09:31:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:5006 write 0x2bb8ef thru 0x2c3ec7 (0x85d9 bytes) 2024-09-12T09:31:37.189 INFO:teuthology.orchestra.run.smithi007.stdout:5007 writesame 0xbbe9711 thru 0xbbe9901 data_size 0x1(0x1f1 bytes) 2024-09-12T09:31:37.197 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.197 INFO:teuthology.orchestra.run.smithi007.stdout:5009 write 0x2531e3b thru 0x2534f11 (0x30d7 bytes) 2024-09-12T09:31:37.212 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.212 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.221 INFO:teuthology.orchestra.run.smithi007.stdout:5010 trunc from 0xdd5525e to 0x6b989fa 2024-09-12T09:31:37.228 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:37.226+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:37.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:37.251 INFO:teuthology.orchestra.run.smithi007.stdout:5011 read 0xd623af thru 0xd6f78a (0xd3dc bytes) 2024-09-12T09:31:37.251 INFO:teuthology.orchestra.run.smithi007.stdout:5012 read 0x42d86c1 thru 0x42e2f01 (0xa841 bytes) 2024-09-12T09:31:37.251 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.251 INFO:teuthology.orchestra.run.smithi007.stdout:5016 punch from 0x4d29a8a to 0x4d2d1f5, (0x376b bytes) 2024-09-12T09:31:37.273 INFO:teuthology.orchestra.run.smithi007.stdout:5017 write 0xb33dbef thru 0xb341f9e (0x43b0 bytes) 2024-09-12T09:31:37.283 INFO:teuthology.orchestra.run.smithi007.stdout:5018 write 0x39527ea thru 0x3953518 (0xd2f bytes) 2024-09-12T09:31:37.291 INFO:teuthology.orchestra.run.smithi007.stdout:5020 trunc from 0xb341f9f to 0xbe46266 2024-09-12T09:31:37.304 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.304 INFO:teuthology.orchestra.run.smithi007.stdout:5024 read 0x22cfb10 thru 0x22d8090 (0x8581 bytes) 2024-09-12T09:31:37.305 INFO:teuthology.orchestra.run.smithi007.stdout:5025 write 0x9ba5f8b thru 0x9ba647b (0x4f1 bytes) 2024-09-12T09:31:37.329 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:37.327+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:37.329 INFO:teuthology.orchestra.run.smithi007.stdout:5026 trunc from 0xbe46266 to 0x1dce5c6 2024-09-12T09:31:37.374 INFO:teuthology.orchestra.run.smithi007.stdout:5027 write 0x292f3a1 thru 0x2935bf2 (0x6852 bytes) 2024-09-12T09:31:37.385 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.386 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.399 INFO:teuthology.orchestra.run.smithi007.stdout:5029 writesame 0x419f452 thru 0x41a1d19 data_size 0x3c(0x28c8 bytes) 2024-09-12T09:31:37.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:37.421 INFO:teuthology.orchestra.run.smithi007.stdout:5030 trunc from 0x41a1d1a to 0xd6b0d3e 2024-09-12T09:31:37.430 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:37.426+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:37.435 INFO:teuthology.orchestra.run.smithi007.stdout:5031 read 0x7f2810b thru 0x7f34265 (0xc15b bytes) 2024-09-12T09:31:37.436 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.436 INFO:teuthology.orchestra.run.smithi007.stdout:5033 punch from 0x29a3db6 to 0x29a5747, (0x1991 bytes) 2024-09-12T09:31:37.440 INFO:teuthology.orchestra.run.smithi007.stdout:5035 trunc from 0xd6b0d3e to 0xb913b36 2024-09-12T09:31:37.454 INFO:teuthology.orchestra.run.smithi007.stdout:5036 punch from 0x49cac02 to 0x49d8f2b, (0xe329 bytes) 2024-09-12T09:31:37.474 INFO:teuthology.orchestra.run.smithi007.stdout:5037 write 0xe0cc1ee thru 0xe0d8f24 (0xcd37 bytes) 2024-09-12T09:31:37.482 INFO:teuthology.orchestra.run.smithi007.stdout:5039 write 0x8a4ea3a thru 0x8a53238 (0x47ff bytes) 2024-09-12T09:31:37.503 INFO:teuthology.orchestra.run.smithi007.stdout:5040 trunc from 0xe0d8f25 to 0x86446e3 2024-09-12T09:31:37.529 INFO:teuthology.orchestra.run.smithi007.stdout:5042 read 0x1c8a616 thru 0x1c90141 (0x5b2c bytes) 2024-09-12T09:31:37.529 INFO:teuthology.orchestra.run.smithi007.stdout:5043 punch from 0x1c990d7 to 0x1ca574c, (0xc675 bytes) 2024-09-12T09:31:37.530 INFO:teuthology.orchestra.run.smithi007.stdout:5045 read 0x4f169f thru 0x4fb9c7 (0xa329 bytes) 2024-09-12T09:31:37.531 INFO:teuthology.orchestra.run.smithi007.stdout:5047 write 0x4331b87 thru 0x433327d (0x16f7 bytes) 2024-09-12T09:31:37.532 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:37.526+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:37.538 INFO:teuthology.orchestra.run.smithi007.stdout:5048 read 0x209c551 thru 0x20a6124 (0x9bd4 bytes) 2024-09-12T09:31:37.552 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.553 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.561 INFO:teuthology.orchestra.run.smithi007.stdout:5051 write 0xc8ccddf thru 0xc8dcb1b (0xfd3d bytes) 2024-09-12T09:31:37.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:37.571 INFO:teuthology.orchestra.run.smithi007.stdout:5052 read 0x4707553 thru 0x47131a8 (0xbc56 bytes) 2024-09-12T09:31:37.571 INFO:teuthology.orchestra.run.smithi007.stdout:5053 punch from 0xb6dfdf8 to 0xb6e8008, (0x8210 bytes) 2024-09-12T09:31:37.571 INFO:teuthology.orchestra.run.smithi007.stdout:5055 read 0xbabd074 thru 0xbac94f5 (0xc482 bytes) 2024-09-12T09:31:37.575 INFO:teuthology.orchestra.run.smithi007.stdout:5056 trunc from 0xc8dcb1c to 0xe9faf2b 2024-09-12T09:31:37.589 INFO:teuthology.orchestra.run.smithi007.stdout:5058 punch from 0xe1a2ce2 to 0xe1a7fdb, (0x52f9 bytes) 2024-09-12T09:31:37.589 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.590 INFO:teuthology.orchestra.run.smithi007.stdout:5060 write 0xdc40c60 thru 0xdc4964b (0x89ec bytes) 2024-09-12T09:31:37.598 INFO:teuthology.orchestra.run.smithi007.stdout:5061 read 0x518bc79 thru 0x518c60f (0x997 bytes) 2024-09-12T09:31:37.603 INFO:teuthology.orchestra.run.smithi007.stdout:5062 trunc from 0xe9faf2b to 0xbd4a061 2024-09-12T09:31:37.627 INFO:teuthology.orchestra.run.smithi007.stdout:5065 read 0x7c5f7ad thru 0x7c64228 (0x4a7c bytes) 2024-09-12T09:31:37.627 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.627 INFO:teuthology.orchestra.run.smithi007.stdout:5067 read 0x6388c32 thru 0x639487a (0xbc49 bytes) 2024-09-12T09:31:37.628 INFO:teuthology.orchestra.run.smithi007.stdout:5068 write 0x6d3212c thru 0x6d33958 (0x182d bytes) 2024-09-12T09:31:37.632 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:37.630+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:37.634 INFO:teuthology.orchestra.run.smithi007.stdout:5069 write 0x49b235d thru 0x49b876c (0x6410 bytes) 2024-09-12T09:31:37.642 INFO:teuthology.orchestra.run.smithi007.stdout:5071 write 0x5349df0 thru 0x5353c31 (0x9e42 bytes) 2024-09-12T09:31:37.650 INFO:teuthology.orchestra.run.smithi007.stdout:5073 writesame 0x43f0f8f thru 0x43f5171 data_size 0x1(0x41e3 bytes) 2024-09-12T09:31:37.678 INFO:teuthology.orchestra.run.smithi007.stdout:5074 writesame 0x553ab96 thru 0x554649d data_size 0x7c(0xb908 bytes) 2024-09-12T09:31:37.687 INFO:teuthology.orchestra.run.smithi007.stdout:5075 write 0x860d112 thru 0x86101b7 (0x30a6 bytes) 2024-09-12T09:31:37.695 INFO:teuthology.orchestra.run.smithi007.stdout:5077 read 0xa625da5 thru 0xa628238 (0x2494 bytes) 2024-09-12T09:31:37.695 INFO:teuthology.orchestra.run.smithi007.stdout:5078 write 0x64fd153 thru 0x6500d52 (0x3c00 bytes) 2024-09-12T09:31:37.704 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.704 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.706 INFO:teuthology.orchestra.run.smithi007.stdout:5080 punch from 0x3c20a1b to 0x3c20c2d, (0x212 bytes) 2024-09-12T09:31:37.706 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.708 INFO:teuthology.orchestra.run.smithi007.stdout:5083 trunc from 0xbd4a061 to 0xd1473dd 2024-09-12T09:31:37.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:37.739 INFO:teuthology.orchestra.run.smithi007.stdout:5084 writesame 0xe5f1e78 thru 0xe5ff4c2 data_size 0x11(0xd64b bytes) 2024-09-12T09:31:37.739 INFO:teuthology.orchestra.run.smithi007.stdout:5085 read 0xd565630 thru 0xd56e7a1 (0x9172 bytes) 2024-09-12T09:31:37.739 INFO:teuthology.orchestra.run.smithi007.stdout:5087 write 0x9e028a4 thru 0x9e04345 (0x1aa2 bytes) 2024-09-12T09:31:37.739 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:37.735+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:37.744 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6] out_osds: [1, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:37.744 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:37.744 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-12T09:31:37.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-12T09:31:37.747 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.747 INFO:teuthology.orchestra.run.smithi007.stdout:5091 write 0x6415922 thru 0x641aeda (0x55b9 bytes) 2024-09-12T09:31:37.753 INFO:teuthology.orchestra.run.smithi007.stdout:5092 write 0x12e9541 thru 0x12ebe61 (0x2921 bytes) 2024-09-12T09:31:37.762 INFO:teuthology.orchestra.run.smithi007.stdout:5093 read 0xc5e1d8a thru 0xc5efbf7 (0xde6e bytes) 2024-09-12T09:31:37.762 INFO:teuthology.orchestra.run.smithi007.stdout:5095 read 0xaa7e384 thru 0xaa82be3 (0x4860 bytes) 2024-09-12T09:31:37.762 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.762 INFO:teuthology.orchestra.run.smithi007.stdout:5097 write 0xa37d0c7 thru 0xa387c4e (0xab88 bytes) 2024-09-12T09:31:37.773 INFO:teuthology.orchestra.run.smithi007.stdout:5098 writesame 0x644f996 thru 0x645a673 data_size 0x1d(0xacde bytes) 2024-09-12T09:31:37.784 INFO:teuthology.orchestra.run.smithi007.stdout:5099 write 0xd2d3e05 thru 0xd2e292f (0xeb2b bytes) 2024-09-12T09:31:37.792 INFO:teuthology.orchestra.run.smithi007.stdout:5100 write 0x257685c thru 0x2585893 (0xf038 bytes) 2024-09-12T09:31:37.804 INFO:teuthology.orchestra.run.smithi007.stdout:5101 punch from 0xd561899 to 0xd56c070, (0xa7d7 bytes) 2024-09-12T09:31:37.804 INFO:teuthology.orchestra.run.smithi007.stdout:5103 write 0xa1a75e thru 0xa1eba4 (0x4447 bytes) 2024-09-12T09:31:37.812 INFO:teuthology.orchestra.run.smithi007.stdout:5104 punch from 0x7e6f149 to 0x7e76570, (0x7427 bytes) 2024-09-12T09:31:37.815 INFO:teuthology.orchestra.run.smithi007.stdout:5105 trunc from 0xe5ff4c3 to 0xc743684 2024-09-12T09:31:37.833 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:37.830+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:37.849 INFO:teuthology.orchestra.run.smithi007.stdout:5106 trunc from 0xc743684 to 0x5b83de0 2024-09-12T09:31:37.875 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:37.875 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:37.879 INFO:teuthology.orchestra.run.smithi007.stdout:5107 read 0x30ddfcb thru 0x30ebacb (0xdb01 bytes) 2024-09-12T09:31:37.879 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:37.898 INFO:teuthology.orchestra.run.smithi007.stdout:5110 write 0xa2db426 thru 0xa2e0ddd (0x59b8 bytes) 2024-09-12T09:31:37.906 INFO:teuthology.orchestra.run.smithi007.stdout:5111 read 0x34d45c1 thru 0x34ddbda (0x961a bytes) 2024-09-12T09:31:37.906 INFO:teuthology.orchestra.run.smithi007.stdout:5112 read 0x1cea5ab thru 0x1cfa384 (0xfdda bytes) 2024-09-12T09:31:37.906 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:37.925 INFO:teuthology.orchestra.run.smithi007.stdout:5114 writesame 0xe48869f thru 0xe493a0a data_size 0x2(0xb36c bytes) 2024-09-12T09:31:37.925 INFO:teuthology.orchestra.run.smithi007.stdout:5115 writesame 0x80bf3fb thru 0x80c015c data_size 0x3(0xd62 bytes) 2024-09-12T09:31:37.934 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:37.930+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:37.946 INFO:teuthology.orchestra.run.smithi007.stdout:5116 trunc from 0xe493a0b to 0x7ab4643 2024-09-12T09:31:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:5117 trunc from 0x7ab4643 to 0xd804315 2024-09-12T09:31:38.008 INFO:teuthology.orchestra.run.smithi007.stdout:5119 trunc from 0xd804315 to 0x6543907 2024-09-12T09:31:38.022 INFO:teuthology.orchestra.run.smithi007.stdout:5121 write 0x5d4d0cd thru 0x5d5113d (0x4071 bytes) 2024-09-12T09:31:38.033 INFO:teuthology.orchestra.run.smithi007.stdout:5123 trunc from 0x6543907 to 0x472e71d 2024-09-12T09:31:38.035 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:38.030+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:38.044 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:38.044 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:38.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:38.063 INFO:teuthology.orchestra.run.smithi007.stdout:5124 read 0x1442447 thru 0x1445519 (0x30d3 bytes) 2024-09-12T09:31:38.063 INFO:teuthology.orchestra.run.smithi007.stdout:5126 read 0x36bbf93 thru 0x36c8a85 (0xcaf3 bytes) 2024-09-12T09:31:38.063 INFO:teuthology.orchestra.run.smithi007.stdout:5127 read 0x105441c thru 0x105cd1e (0x8903 bytes) 2024-09-12T09:31:38.064 INFO:teuthology.orchestra.run.smithi007.stdout:5128 punch from 0x33dfe3c to 0x33e1551, (0x1715 bytes) 2024-09-12T09:31:38.082 INFO:teuthology.orchestra.run.smithi007.stdout:5130 writesame 0x88f084a thru 0x88f698f data_size 0x1(0x6146 bytes) 2024-09-12T09:31:38.083 INFO:teuthology.orchestra.run.smithi007.stdout:5131 punch from 0x882f660 to 0x883ab77, (0xb517 bytes) 2024-09-12T09:31:38.083 INFO:teuthology.orchestra.run.smithi007.stdout:5134 read 0x6a581fe thru 0x6a66554 (0xe357 bytes) 2024-09-12T09:31:38.083 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.097 INFO:teuthology.orchestra.run.smithi007.stdout:5139 writesame 0xadcdeee thru 0xaddd4e9 data_size 0x34(0xf5fc bytes) 2024-09-12T09:31:38.097 INFO:teuthology.orchestra.run.smithi007.stdout:5140 read 0x368a2c0 thru 0x368e68f (0x43d0 bytes) 2024-09-12T09:31:38.097 INFO:teuthology.orchestra.run.smithi007.stdout:5141 read 0x78c975a thru 0x78d57b4 (0xc05b bytes) 2024-09-12T09:31:38.097 INFO:teuthology.orchestra.run.smithi007.stdout:5142 read 0x75263a thru 0x761ecb (0xf892 bytes) 2024-09-12T09:31:38.098 INFO:teuthology.orchestra.run.smithi007.stdout:5143 write 0xa014be5 thru 0xa022d63 (0xe17f bytes) 2024-09-12T09:31:38.100 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:31:38.099+0000 7f3454e55640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2024-09-12T09:31:38.106 INFO:teuthology.orchestra.run.smithi007.stdout:5145 read 0xa1e7e45 thru 0xa1ee074 (0x6230 bytes) 2024-09-12T09:31:38.123 INFO:teuthology.orchestra.run.smithi007.stdout:5147 writesame 0xeb74906 thru 0xeb7dce1 data_size 0x2(0x93dc bytes) 2024-09-12T09:31:38.123 INFO:teuthology.orchestra.run.smithi007.stdout:5149 write 0xb710463 thru 0xb7202fc (0xfe9a bytes) 2024-09-12T09:31:38.130 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.130 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.130 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.130 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:5159 read 0x9babfac thru 0x9bb733e (0xb393 bytes) 2024-09-12T09:31:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.131 INFO:teuthology.orchestra.run.smithi007.stdout:5161 write 0x28ccccb thru 0x28d5aca (0x8e00 bytes) 2024-09-12T09:31:38.136 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:38.134+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:38.138 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.138 INFO:teuthology.orchestra.run.smithi007.stdout:5165 read 0xb656d75 thru 0xb6587d8 (0x1a64 bytes) 2024-09-12T09:31:38.138 INFO:teuthology.orchestra.run.smithi007.stdout:5166 read 0xc632008 thru 0xc63768d (0x5686 bytes) 2024-09-12T09:31:38.139 INFO:teuthology.orchestra.run.smithi007.stdout:5167 punch from 0x4a4e to 0xa377, (0x5929 bytes) 2024-09-12T09:31:38.139 INFO:teuthology.orchestra.run.smithi007.stdout:5168 write 0x7e7ef16 thru 0x7e83b36 (0x4c21 bytes) 2024-09-12T09:31:38.147 INFO:teuthology.orchestra.run.smithi007.stdout:5169 read 0xe328c02 thru 0xe32a1ff (0x15fe bytes) 2024-09-12T09:31:38.148 INFO:teuthology.orchestra.run.smithi007.stdout:5170 write 0xb96c15c thru 0xb977a0f (0xb8b4 bytes) 2024-09-12T09:31:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:5171 read 0x1404186 thru 0x140b9a4 (0x781f bytes) 2024-09-12T09:31:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:5172 write 0xe20c74b thru 0xe21653a (0x9df0 bytes) 2024-09-12T09:31:38.162 INFO:teuthology.orchestra.run.smithi007.stdout:5173 writesame 0xa71de88 thru 0xa7215cc data_size 0x1(0x3745 bytes) 2024-09-12T09:31:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:5174 read 0xa8696c2 thru 0xa871195 (0x7ad4 bytes) 2024-09-12T09:31:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:5176 read 0x2f13fb2 thru 0x2f225b4 (0xe603 bytes) 2024-09-12T09:31:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:5177 read 0x70894d0 thru 0x7096d97 (0xd8c8 bytes) 2024-09-12T09:31:38.187 INFO:teuthology.orchestra.run.smithi007.stdout:5178 write 0x25dcd50 thru 0x25ea967 (0xdc18 bytes) 2024-09-12T09:31:38.195 INFO:teuthology.orchestra.run.smithi007.stdout:5179 read 0xa4b2e00 thru 0xa4b3137 (0x338 bytes) 2024-09-12T09:31:38.195 INFO:teuthology.orchestra.run.smithi007.stdout:5180 read 0xa0d69a3 thru 0xa0dd96a (0x6fc8 bytes) 2024-09-12T09:31:38.196 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.196 INFO:teuthology.orchestra.run.smithi007.stdout:5184 write 0x4e42fce thru 0x4e45177 (0x21aa bytes) 2024-09-12T09:31:38.199 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:38.200 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:38.204 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.204 INFO:teuthology.orchestra.run.smithi007.stdout:5186 punch from 0xd39bb14 to 0xd3aa778, (0xec64 bytes) 2024-09-12T09:31:38.204 INFO:teuthology.orchestra.run.smithi007.stdout:5190 punch from 0xc200c5e to 0xc20871a, (0x7abc bytes) 2024-09-12T09:31:38.204 INFO:teuthology.orchestra.run.smithi007.stdout:5191 write 0xa3101df thru 0xa31dc16 (0xda38 bytes) 2024-09-12T09:31:38.209 INFO:teuthology.orchestra.run.smithi007.stdout:5192 write 0x36c938f thru 0x36d62c6 (0xcf38 bytes) 2024-09-12T09:31:38.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:38.220 INFO:teuthology.orchestra.run.smithi007.stdout:5193 write 0xbde0eac thru 0xbde915b (0x82b0 bytes) 2024-09-12T09:31:38.226 INFO:teuthology.orchestra.run.smithi007.stdout:5194 write 0x76ab69b thru 0x76b731a (0xbc80 bytes) 2024-09-12T09:31:38.233 INFO:teuthology.orchestra.run.smithi007.stdout:5195 write 0x55f5b05 thru 0x560041a (0xa916 bytes) 2024-09-12T09:31:38.237 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:38.235+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:38.243 INFO:teuthology.orchestra.run.smithi007.stdout:5196 read 0x9be633d thru 0x9bf4149 (0xde0d bytes) 2024-09-12T09:31:38.243 INFO:teuthology.orchestra.run.smithi007.stdout:5197 write 0x3f692a7 thru 0x3f72f9f (0x9cf9 bytes) 2024-09-12T09:31:38.253 INFO:teuthology.orchestra.run.smithi007.stdout:5199 read 0x9900c29 thru 0x9909ca8 (0x9080 bytes) 2024-09-12T09:31:38.253 INFO:teuthology.orchestra.run.smithi007.stdout:5200 punch from 0xa530318 to 0xa533fa6, (0x3c8e bytes) 2024-09-12T09:31:38.253 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.253 INFO:teuthology.orchestra.run.smithi007.stdout:5208 write 0x27997d4 thru 0x27a6471 (0xcc9e bytes) 2024-09-12T09:31:38.256 INFO:teuthology.orchestra.run.smithi007.stdout:5209 write 0x2c1538d thru 0x2c15a1d (0x691 bytes) 2024-09-12T09:31:38.263 INFO:teuthology.orchestra.run.smithi007.stdout:5210 writesame 0x6417d49 thru 0x641f83c data_size 0x3d(0x7af4 bytes) 2024-09-12T09:31:38.270 INFO:teuthology.orchestra.run.smithi007.stdout:5211 write 0x9bf569c thru 0x9c029e4 (0xd349 bytes) 2024-09-12T09:31:38.282 INFO:teuthology.orchestra.run.smithi007.stdout:5212 write 0x88caef7 thru 0x88d51b7 (0xa2c1 bytes) 2024-09-12T09:31:38.290 INFO:teuthology.orchestra.run.smithi007.stdout:5215 punch from 0x612a19c to 0x612bf81, (0x1de5 bytes) 2024-09-12T09:31:38.291 INFO:teuthology.orchestra.run.smithi007.stdout:5216 read 0x603e945 thru 0x604bab4 (0xd170 bytes) 2024-09-12T09:31:38.291 INFO:teuthology.orchestra.run.smithi007.stdout:5217 read 0x5b6a129 thru 0x5b72fe1 (0x8eb9 bytes) 2024-09-12T09:31:38.291 INFO:teuthology.orchestra.run.smithi007.stdout:5218 read 0x122bd10 thru 0x1234445 (0x8736 bytes) 2024-09-12T09:31:38.291 INFO:teuthology.orchestra.run.smithi007.stdout:5219 punch from 0x5ebc19e to 0x5ebdd81, (0x1be3 bytes) 2024-09-12T09:31:38.305 INFO:teuthology.orchestra.run.smithi007.stdout:5220 trunc from 0xeb7dce2 to 0x6d41d3b 2024-09-12T09:31:38.338 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:38.334+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:38.347 INFO:teuthology.orchestra.run.smithi007.stdout:5221 trunc from 0x6d41d3b to 0xb080088 2024-09-12T09:31:38.361 INFO:teuthology.orchestra.run.smithi007.stdout:5223 write 0x6be7136 thru 0x6bf45be (0xd489 bytes) 2024-09-12T09:31:38.361 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:38.361 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:38.369 INFO:teuthology.orchestra.run.smithi007.stdout:5225 punch from 0x3eb8e9b to 0x3ec6d6b, (0xded0 bytes) 2024-09-12T09:31:38.369 INFO:teuthology.orchestra.run.smithi007.stdout:5226 punch from 0x32e44b6 to 0x32f0807, (0xc351 bytes) 2024-09-12T09:31:38.369 INFO:teuthology.orchestra.run.smithi007.stdout:5227 read 0x6b4ac6e thru 0x6b54a9e (0x9e31 bytes) 2024-09-12T09:31:38.377 INFO:teuthology.orchestra.run.smithi007.stdout:5228 trunc from 0xb080088 to 0xe7d83e9 2024-09-12T09:31:38.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:5229 read 0x5f74d47 thru 0x5f78252 (0x350c bytes) 2024-09-12T09:31:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:5230 read 0x659da21 thru 0x65a55b0 (0x7b90 bytes) 2024-09-12T09:31:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:5231 write 0x546f5b7 thru 0x54752a6 (0x5cf0 bytes) 2024-09-12T09:31:38.400 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.400 INFO:teuthology.orchestra.run.smithi007.stdout:5233 write 0xba80c57 thru 0xba841a2 (0x354c bytes) 2024-09-12T09:31:38.407 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.407 INFO:teuthology.orchestra.run.smithi007.stdout:5236 read 0xd4f1305 thru 0xd4f5d21 (0x4a1d bytes) 2024-09-12T09:31:38.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:38.411 INFO:teuthology.orchestra.run.smithi007.stdout:5237 trunc from 0xe7d83e9 to 0xc167728 2024-09-12T09:31:38.423 INFO:teuthology.orchestra.run.smithi007.stdout:5239 write 0x6eb702e thru 0x6ec25bb (0xb58e bytes) 2024-09-12T09:31:38.434 INFO:teuthology.orchestra.run.smithi007.stdout:5240 punch from 0x2046697 to 0x2046d3b, (0x6a4 bytes) 2024-09-12T09:31:38.439 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:38.435+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:38.447 INFO:teuthology.orchestra.run.smithi007.stdout:5243 trunc from 0xc167728 to 0x3e96613 2024-09-12T09:31:38.481 INFO:teuthology.orchestra.run.smithi007.stdout:5244 read 0x1b492a1 thru 0x1b4a15e (0xebe bytes) 2024-09-12T09:31:38.481 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.481 INFO:teuthology.orchestra.run.smithi007.stdout:5246 read 0x332cbd thru 0x33a891 (0x7bd5 bytes) 2024-09-12T09:31:38.481 INFO:teuthology.orchestra.run.smithi007.stdout:5248 read 0x2c8e624 thru 0x2c9d50d (0xeeea bytes) 2024-09-12T09:31:38.510 INFO:teuthology.orchestra.run.smithi007.stdout:5249 writesame 0xd4000ff thru 0xd407708 data_size 0x1d(0x760a bytes) 2024-09-12T09:31:38.510 INFO:teuthology.orchestra.run.smithi007.stdout:5250 write 0x1b5e735 thru 0x1b62c5d (0x4529 bytes) 2024-09-12T09:31:38.518 INFO:teuthology.orchestra.run.smithi007.stdout:5251 trunc from 0xd407709 to 0xd64a323 2024-09-12T09:31:38.528 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:38.528 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:38.534 INFO:teuthology.orchestra.run.smithi007.stdout:5253 read 0xd4df410 thru 0xd4e111a (0x1d0b bytes) 2024-09-12T09:31:38.540 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:38.539+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:38.543 INFO:teuthology.orchestra.run.smithi007.stdout:5256 trunc from 0xd64a323 to 0x8384f10 2024-09-12T09:31:38.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:38.556 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:38.556 INFO:teuthology.orchestra.run.smithi007.stdout:5258 read 0x4f8489c thru 0x4f8628b (0x19f0 bytes) 2024-09-12T09:31:38.556 INFO:teuthology.orchestra.run.smithi007.stdout:5261 read 0x2eb5fb3 thru 0x2ec2c57 (0xcca5 bytes) 2024-09-12T09:31:38.557 INFO:teuthology.orchestra.run.smithi007.stdout:5262 trunc from 0x8384f10 to 0x871b0a6 2024-09-12T09:31:38.567 INFO:teuthology.orchestra.run.smithi007.stdout:5264 read 0x50f968d thru 0x510735c (0xdcd0 bytes) 2024-09-12T09:31:38.570 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:38.570 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:38.583 INFO:teuthology.orchestra.run.smithi007.stdout:5266 write 0xc28391c thru 0xc285aa6 (0x218b bytes) 2024-09-12T09:31:38.588 INFO:teuthology.orchestra.run.smithi007.stdout:5267 clone 44 order 25 su 524288 sc 10 2024-09-12T09:31:38.588 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:38.588 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:38.605 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:38.605 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:38.622 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:38.640 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:38.638+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:38.642 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:38.661 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:38.683 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:38.683 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:38.683 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:38.683 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:38.684 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:38.684 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:38.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:38.704 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:38.741 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:38.738+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:38.742 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.1. 2024-09-12T09:31:38.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:31:38.772 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-12T09:31:38.841 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:38.838+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:38.868 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:38.869 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:38.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:38.968 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:38.966+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:39.043 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:39.039+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:39.050 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.758654461999996, 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.299158233, 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.925741282000004, 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.925694035000006, 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.925400959000001, 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.924839027000004, 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.924787663999993, 2024-09-12T09:31:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.924214784, 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.924062484999993, 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.923705394999999, 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.923080424999995, 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.923029088999996, 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.922234298999996, 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.922159409000002, 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.922107088999994, 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.922058491000001, 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.921804026000004, 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.918349735999996, 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.860781188999994, 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 66.389509951999997, 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.150 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:39.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:39.152 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:39.146+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:39.252 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:39.250+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:39.323 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:39.323 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:39.323 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:39.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:39.353 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:39.351+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:39.454 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:39.451+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:39.506 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:39.506 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:39.520 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:39.555 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:39.551+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.233578413, 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.233483151, 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.229798527, 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.22799343699999, 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.227446762, 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.226678319, 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.226267951, 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.22612877500001, 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.218349964, 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.21598595, 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.205234057, 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.205037666, 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.203543246, 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 86.503300385000003, 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 86.503277967000002, 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 86.503254826000003, 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 86.503229867000002, 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 76.459878352999993, 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.843400310999996, 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.843301021000002, 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:39.758 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:31:39.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:39.760 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:39.755+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:39.861 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:39.858+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:39.890 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:39.890 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:39.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:39.996 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:39.994+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:40.062 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:40.058+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:40.074 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:40.074 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:40.074 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:40.074 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:40.074 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:40.089 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:40.176 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:40.174+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61879999600001, 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:40.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618750388, 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.264 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:40.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618716646, 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:40.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:40.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618669608, 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:40.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61861176799999, 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:40.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:40.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618552681, 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:40.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:40.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618518494, 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:40.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61848715799999, 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61845711700001, 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618425872, 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618384103, 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618352637, 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618305097, 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61827206, 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61822703999999, 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618194382, 2024-09-12T09:31:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.618162567, 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61810465400001, 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61806267199999, 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.61802668999999, 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.354 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:40.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:40.356 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:40.350+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:40.456 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:40.455+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:40.522 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:40.522 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:40.522 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:40.522 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:40.537 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:40.557 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:40.555+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:40.654 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:40.654 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:40.655 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:40.658 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:40.655+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:40.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:40.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:40.772 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:40.771+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.794 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.577684618999999, 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:40.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:31:40.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.575561001000001, 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:40.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:31:40.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.575541174000001, 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:31:40.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:31:40.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.573446141000002, 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:40.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571411533999999, 2024-09-12T09:31:40.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:40.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:31:40.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:31:40.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571372817, 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:31:40.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:31:40.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571335832999999, 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:31:40.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:31:40.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:40.835 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571321448999999, 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:40.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:31:40.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571300458, 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:31:40.841 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:31:40.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571288986999999, 2024-09-12T09:31:40.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:31:40.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:31:40.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.571263502000001, 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.569076222, 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:31:40.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:31:40.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:31:40.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.569065854000002, 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:31:40.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:31:40.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.569039887999999, 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:40.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:31:40.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.567348066000001, 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:31:40.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.567333510000001, 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:40.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:31:40.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.56731314, 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:31:40.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:31:40.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.567274512000001, 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:31:40.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.567259989, 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:31:40.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:40.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.567249574000002, 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:40.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:40.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:40.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:40.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:40.914 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:40.915 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:40.915 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:40.916 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:40.916 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:40.917 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:40.910+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:40.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:40.922 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:40.940 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:40.940 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:40.961 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:40.982 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:40.983 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:41.004 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:41.016 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:41.015+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:41.027 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:41.037 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:41.037 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:41.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:41.117 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:41.114+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:41.217 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:41.215+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:41.237 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:41.237 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:41.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:41.344 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:41.343+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:41.419 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:41.414+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:41.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:41.544 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:41.543+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:41.581 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-12T09:31:41.621 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:41.619+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:41.643 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:41.644 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:41.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:41.722 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:41.718+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:41.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:31:41.828 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:41.827+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:41.865 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:41.865 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:41.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:41.928 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:41.927+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:42.024 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:42.022+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:42.136 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:42.135+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:42.225 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:42.223+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:42.237 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.237 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:42.348 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:42.347+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:42.427 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:42.423+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:42.432 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.432 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:42.528 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:42.526+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:42.629 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:42.626+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:42.632 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.632 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:42.665 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:31:42.730 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:42.726+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:42.791 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.791 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:42.852 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:42.851+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:42.928 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:42.928 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:42.931 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:42.926+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:42.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:42.984 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:42.984 INFO:teuthology.orchestra.run.smithi007.stdout:5269 write 0x96a51f5 thru 0x96a8f6f (0x3d7b bytes) 2024-09-12T09:31:43.032 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:43.031+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:43.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:43.111 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:43.112 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:43.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:43.132 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:43.131+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:43.187 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:43.188 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:43.204 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:43.204 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:43.221 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:43.222 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:43.233 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:43.230+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:43.238 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:43.258 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:43.259 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:43.261 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:43.261 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:43.261 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:43.261 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:43.275 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:43.278 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:43.279 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:43.298 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:43.318 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:43.318 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:43.364 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:43.362+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:43.436 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:43.434+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:43.439 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:43.439 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:43.439 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:43.439 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:43.440 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:43.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:43.540 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:43.538+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:43.625 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:43.625 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.333540280999998, 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:43.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.874044052000002, 2024-09-12T09:31:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:43.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:43.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:43.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.500627101000006, 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:43.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:43.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.500579853999994, 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:43.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.500286778000003, 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:43.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.499724846000007, 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:43.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:43.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:43.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.499673482999995, 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:43.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.499100603000002, 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:43.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:43.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.498948303999995, 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:43.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:43.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.498591214000001, 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:43.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.497966243999997, 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:43.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:43.677 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.497914907999998, 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:43.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:43.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.497120117999998, 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.497045228000005, 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:43.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.496992907999996, 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:43.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:43.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:43.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.496944310000003, 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:43.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.496689845000006, 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:43.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:43.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.493235554999998, 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:43.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:43.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.435667007999996, 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:43.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:43.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.964395771, 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:43.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:43.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:43.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:43.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:43.723 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:43.724 INFO:teuthology.orchestra.run.smithi007.stdout:5270 punch from 0x980128b to 0x980ef52, (0xdcc7 bytes) 2024-09-12T09:31:43.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:43.725 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:43.722+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:43.725 INFO:teuthology.orchestra.run.smithi007.stdout:5274 writesame 0xebef973 thru 0xebf13cb data_size 0x13(0x1a59 bytes) 2024-09-12T09:31:43.728 INFO:teuthology.orchestra.run.smithi007.stdout:5275 write 0x7f7492c thru 0x7f8250d (0xdbe2 bytes) 2024-09-12T09:31:43.773 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:43.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:43.774 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T09:31:43.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:31:43.779 INFO:teuthology.orchestra.run.smithi007.stdout:5276 trunc from 0xebf13cc to 0x3483de3 2024-09-12T09:31:43.817 INFO:teuthology.orchestra.run.smithi007.stdout:5277 trunc from 0x3483de3 to 0x1344aa2 2024-09-12T09:31:43.825 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:43.822+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:43.862 INFO:teuthology.orchestra.run.smithi007.stdout:5278 write 0xa0259d0 thru 0xa029331 (0x3962 bytes) 2024-09-12T09:31:43.865 INFO:teuthology.orchestra.run.smithi007.stdout:5279 write 0x30d56d thru 0x317c5f (0xa6f3 bytes) 2024-09-12T09:31:43.886 INFO:teuthology.orchestra.run.smithi007.stdout:5280 trunc from 0xa029332 to 0x1acf117 2024-09-12T09:31:43.902 INFO:teuthology.orchestra.run.smithi007.stdout:5281 punch from 0x133ae7f to 0x133de28, (0x2fa9 bytes) 2024-09-12T09:31:43.903 INFO:teuthology.orchestra.run.smithi007.stdout:5282 trunc from 0x1acf117 to 0x12f6cbe 2024-09-12T09:31:43.908 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:43.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:43.908 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:43.908 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:43.917 INFO:teuthology.orchestra.run.smithi007.stdout:5284 read 0x11de07f thru 0x11dee67 (0xde9 bytes) 2024-09-12T09:31:43.924 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:43.947 INFO:teuthology.orchestra.run.smithi007.stdout:5285 write 0xe256a28 thru 0xe25b12d (0x4706 bytes) 2024-09-12T09:31:43.949 INFO:teuthology.orchestra.run.smithi007.stdout:5286 read 0x1f2aeba thru 0x1f308c9 (0x5a10 bytes) 2024-09-12T09:31:43.949 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:43.967 INFO:teuthology.orchestra.run.smithi007.stdout:5288 trunc from 0xe25b12e to 0x1747236 2024-09-12T09:31:43.969 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:43.966+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:44.000 INFO:teuthology.orchestra.run.smithi007.stdout:5289 write 0x989252a thru 0x989265e (0x135 bytes) 2024-09-12T09:31:44.002 INFO:teuthology.orchestra.run.smithi007.stdout:5291 write 0xa93e93 thru 0xaa0437 (0xc5a5 bytes) 2024-09-12T09:31:44.018 INFO:teuthology.orchestra.run.smithi007.stdout:5292 write 0xb2b35e2 thru 0xb2bacf3 (0x7712 bytes) 2024-09-12T09:31:44.026 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:44.022+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:44.027 INFO:teuthology.orchestra.run.smithi007.stdout:5293 trunc from 0xb2bacf4 to 0x7b15626 2024-09-12T09:31:44.052 INFO:teuthology.orchestra.run.smithi007.stdout:5294 write 0x86e661f thru 0x86ee406 (0x7de8 bytes) 2024-09-12T09:31:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:5296 write 0x792d89a thru 0x792f5e1 (0x1d48 bytes) 2024-09-12T09:31:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:5297 trunc from 0x86ee407 to 0x60c714e 2024-09-12T09:31:44.077 INFO:teuthology.orchestra.run.smithi007.stdout:5298 write 0x97507b thru 0x9813f3 (0xc379 bytes) 2024-09-12T09:31:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:5299 punch from 0x5ee2709 to 0x5ee898d, (0x6284 bytes) 2024-09-12T09:31:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:44.081 INFO:teuthology.orchestra.run.smithi007.stdout:5301 read 0x251f496 thru 0x251fa64 (0x5cf bytes) 2024-09-12T09:31:44.082 INFO:teuthology.orchestra.run.smithi007.stdout:5303 write 0x5f05d6b thru 0x5f08a01 (0x2c97 bytes) 2024-09-12T09:31:44.101 INFO:teuthology.orchestra.run.smithi007.stdout:5305 write 0xae9928d thru 0xaea5ce2 (0xca56 bytes) 2024-09-12T09:31:44.101 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:44.102 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:44.102 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:44.102 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:44.102 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:44.104 INFO:teuthology.orchestra.run.smithi007.stdout:5306 read 0x9f2084d thru 0x9f256a3 (0x4e57 bytes) 2024-09-12T09:31:44.105 INFO:teuthology.orchestra.run.smithi007.stdout:5307 write 0x52441b thru 0x52834f (0x3f35 bytes) 2024-09-12T09:31:44.107 INFO:teuthology.orchestra.run.smithi007.stdout:5308 read 0x58d43ee thru 0x58dea8f (0xa6a2 bytes) 2024-09-12T09:31:44.108 INFO:teuthology.orchestra.run.smithi007.stdout:5309 read 0x54fb7a9 thru 0x55097ca (0xe022 bytes) 2024-09-12T09:31:44.109 INFO:teuthology.orchestra.run.smithi007.stdout:5310 write 0xf95e14 thru 0xfa549f (0xf68c bytes) 2024-09-12T09:31:44.113 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:44.113 INFO:teuthology.orchestra.run.smithi007.stdout:5314 write 0x372dcf8 thru 0x3730ed1 (0x31da bytes) 2024-09-12T09:31:44.116 INFO:teuthology.orchestra.run.smithi007.stdout:5315 punch from 0x1d486d to 0x1da460, (0x5bf3 bytes) 2024-09-12T09:31:44.116 INFO:teuthology.orchestra.run.smithi007.stdout:5316 write 0x198df71 thru 0x1990d22 (0x2db2 bytes) 2024-09-12T09:31:44.117 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:5317 read 0x458ca38 thru 0x4591b19 (0x50e2 bytes) 2024-09-12T09:31:44.119 INFO:teuthology.orchestra.run.smithi007.stdout:5320 writesame 0x78e0259 thru 0x78e2974 data_size 0x4(0x271c bytes) 2024-09-12T09:31:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:5321 read 0x662b6 thru 0x684e3 (0x222e bytes) 2024-09-12T09:31:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:5322 punch from 0x4cffd45 to 0x4d09328, (0x95e3 bytes) 2024-09-12T09:31:44.122 INFO:teuthology.orchestra.run.smithi007.stdout:5323 read 0x3900e76 thru 0x390b354 (0xa4df bytes) 2024-09-12T09:31:44.123 INFO:teuthology.orchestra.run.smithi007.stdout:5324 writesame 0x2753b5a thru 0x2755619 data_size 0x4(0x1ac0 bytes) 2024-09-12T09:31:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:5326 punch from 0x4dfc5d6 to 0x4e0264c, (0x6076 bytes) 2024-09-12T09:31:44.126 INFO:teuthology.orchestra.run.smithi007.stdout:5327 write 0x97c189b thru 0x97cee7d (0xd5e3 bytes) 2024-09-12T09:31:44.127 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:44.123+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:5328 punch from 0x2b05b23 to 0x2b0d5b8, (0x7a95 bytes) 2024-09-12T09:31:44.130 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 129, 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:31:44.132 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:31:43.666029+0000", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:29:16.082807+0000", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:38.740838+0000", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:31:44.133 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:31:44.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:44.135 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:44.136 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:44.137 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:44.138 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:44.139 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.140 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:44.142 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:31:44.143 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:44.144 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:44.145 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:44.145 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:44.147 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:44.148 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:44.149 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:44.150 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.9900000095367432, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.9900000095367432, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:44.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:31:44.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:31:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:31:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:31:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:31:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:31:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:31:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "124", 2024-09-12T09:31:44.155 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 46, 2024-09-12T09:31:44.156 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 124, 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:31:44.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:31:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:31:44.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:31:44.160 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:31:44.161 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.9900000095367432, 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.9900000095367432, 2024-09-12T09:31:44.162 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:31:44.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.164 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 103, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:31:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:31:44.169 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.170 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 128, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.178 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 128, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.183 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:31:44.184 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.185 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 128, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.191 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.192 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.192 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:44.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 126, 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.201 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 124, 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:31:44.206 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.207 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 115, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:31:44.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:31:44.213 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 124, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.216 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:31:44.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:31:44.219 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-12T09:31:44.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.224 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.225 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:31:44.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.2", 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:31:44.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1a", 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:31:44.229 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:31:44.230 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:31:44.231 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:31:44.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:31:44.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:44.233 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:44.235 INFO:teuthology.orchestra.run.smithi007.stdout:5330 write 0xc66e9f0 thru 0xc67ac7c (0xc28d bytes) 2024-09-12T09:31:44.235 INFO:teuthology.orchestra.run.smithi007.stdout:5331 write 0x7ab4138 thru 0x7ab710a (0x2fd3 bytes) 2024-09-12T09:31:44.237 INFO:tasks.thrashosds.thrasher:j is {'epoch': 129, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:31:43.666029+0000', 'last_up_change': '2024-09-12T09:29:16.082807+0000', 'last_in_change': '2024-09-12T09:31:38.740838+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 115, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [6, 7]}, {'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:31:44.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:31:44.238 INFO:teuthology.orchestra.run.smithi007.stdout:5333 writesame 0xd160efa thru 0xd16701e data_size 0x1(0x6125 bytes) 2024-09-12T09:31:44.238 INFO:teuthology.orchestra.run.smithi007.stdout:5335 writesame 0xc7c27ea thru 0xc7c98a6 data_size 0x13(0x70bd bytes) 2024-09-12T09:31:44.239 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:44.239 INFO:teuthology.orchestra.run.smithi007.stdout:5337 clone 45 order 22 su 1048576 sc 13 2024-09-12T09:31:44.239 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:44.230+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:44.335 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:44.335+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:44.366 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.954352738, 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.954257476, 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.95057285199999, 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.948767762, 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.94822108699999, 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.94745264399999, 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.947042276, 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.9469031, 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.93912428900001, 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.936760275, 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.92600838200001, 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.925811991, 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.924317571, 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.224074709999996, 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.224052291999996, 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.224029150999996, 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.224004191999995, 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.180652678000001, 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.564174636000004, 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.564075345999996, 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.469 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:44.471 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:44.471 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:44.467+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:44.571 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:44.567+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:31:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:31:44.599 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:44.600 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:44.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:44.617 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-12T09:31:44.009099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1729677804,"num_objects":1805,"num_object_clones":477,"num_object_copies":3610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1805,"num_whiteouts":639,"num_read":223878,"num_read_kb":1430625,"num_write":18524,"num_write_kb":791833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2558,"num_bytes_recovered":617056750,"num_keys_recovered":482,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14327,"ondisk_log_size":14327,"up":82,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":431188,"kb_used_data":93484,"kb_used_omap":10,"kb_used_meta":337525,"kb_avail":660171692,"statfs":{"total":676457349120,"available":676015812608,"internally_reserved":0,"allocated":95727616,"data_stored":1096185610,"data_compressed":17762840,"data_compressed_allocated":74305536,"data_compressed_original":1084197664,"omap_allocated":11157,"internal_metadata":345625707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7336387,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":2576,"num_read_kb":18944,"num_write":466,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001130"},"pg_stats":[{"pgid":"3.1f","version":"123'658","reported_seq":5939,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.761078+0000","last_change":"2024-09-12T09:31:12.221541+0000","last_active":"2024-09-12T09:31:40.761078+0000","last_peered":"2024-09-12T09:31:40.761078+0000","last_clean":"2024-09-12T09:31:40.761078+0000","last_became_active":"2024-09-12T09:31:12.219747+0000","last_became_peered":"2024-09-12T09:31:12.219747+0000","last_unstale":"2024-09-12T09:31:40.761078+0000","last_undegraded":"2024-09-12T09:31:40.761078+0000","last_fullsized":"2024-09-12T09:31:40.761078+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:53:38.622635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":95673041,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":15,"num_read":7667,"num_read_kb":94780,"num_write":874,"num_write_kb":47326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":37755293,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"121'544","reported_seq":4923,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.754242+0000","last_change":"2024-09-12T09:31:25.691962+0000","last_active":"2024-09-12T09:31:40.754242+0000","last_peered":"2024-09-12T09:31:40.754242+0000","last_clean":"2024-09-12T09:31:40.754242+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:31:40.754242+0000","last_undegraded":"2024-09-12T09:31:40.754242+0000","last_fullsized":"2024-09-12T09:31:40.754242+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:13.039185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187100000000001,"stat_sum":{"num_bytes":45391979,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":16,"num_read":7007,"num_read_kb":35661,"num_write":688,"num_write_kb":34143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"123'301","reported_seq":6930,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.760994+0000","last_change":"2024-09-12T09:29:36.686146+0000","last_active":"2024-09-12T09:31:40.760994+0000","last_peered":"2024-09-12T09:31:40.760994+0000","last_clean":"2024-09-12T09:31:40.760994+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:31:40.760994+0000","last_undegraded":"2024-09-12T09:31:40.760994+0000","last_fullsized":"2024-09-12T09:31:40.760994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:02.722315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011843890000000001,"stat_sum":{"num_bytes":54723371,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":24,"num_read":7375,"num_read_kb":75659,"num_write":296,"num_write_kb":41677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"123'388","reported_seq":4467,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.762035+0000","last_change":"2024-09-12T09:29:36.687808+0000","last_active":"2024-09-12T09:31:40.762035+0000","last_peered":"2024-09-12T09:31:40.762035+0000","last_clean":"2024-09-12T09:31:40.762035+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:31:40.762035+0000","last_undegraded":"2024-09-12T09:31:40.762035+0000","last_fullsized":"2024-09-12T09:31:40.762035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:55.165751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025888840000000001,"stat_sum":{"num_bytes":25842340,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":5038,"num_read_kb":35078,"num_write":441,"num_write_kb":12442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"123'298","reported_seq":5721,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.782550+0000","last_change":"2024-09-12T09:31:12.230274+0000","last_active":"2024-09-12T09:31:41.782550+0000","last_peered":"2024-09-12T09:31:41.782550+0000","last_clean":"2024-09-12T09:31:41.782550+0000","last_became_active":"2024-09-12T09:31:12.228841+0000","last_became_peered":"2024-09-12T09:31:12.228841+0000","last_unstale":"2024-09-12T09:31:41.782550+0000","last_undegraded":"2024-09-12T09:31:41.782550+0000","last_fullsized":"2024-09-12T09:31:41.782550+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:13.197690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025472099999999998,"stat_sum":{"num_bytes":35654661,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":5903,"num_read_kb":19810,"num_write":310,"num_write_kb":5980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":4210634,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"123'463","reported_seq":8000,"reported_epoch":126,"state":"activating+undersized+degraded+remapped","last_fresh":"2024-09-12T09:31:40.753528+0000","last_change":"2024-09-12T09:31:41.064960+0000","last_active":"2024-09-12T09:31:38.745872+0000","last_peered":"2024-09-12T09:31:38.155203+0000","last_clean":"2024-09-12T09:31:38.155203+0000","last_became_active":"2024-09-12T09:30:46.978472+0000","last_became_peered":"2024-09-12T09:30:46.978472+0000","last_unstale":"2024-09-12T09:31:40.753528+0000","last_undegraded":"2024-09-12T09:31:40.753528+0000","last_fullsized":"2024-09-12T09:31:40.751723+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001332878,"stat_sum":{"num_bytes":43321857,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":8161,"num_read_kb":30590,"num_write":529,"num_write_kb":37365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2195003,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","4"],"object_location_counts":[{"shards":"2,4","objects":56}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"123'318","reported_seq":3094,"reported_epoch":126,"state":"activating+undersized+remapped","last_fresh":"2024-09-12T09:31:40.751359+0000","last_change":"2024-09-12T09:31:40.753856+0000","last_active":"2024-09-12T09:31:39.752418+0000","last_peered":"2024-09-12T09:31:37.341574+0000","last_clean":"2024-09-12T09:31:37.341574+0000","last_became_active":"2024-09-12T09:30:43.967238+0000","last_became_peered":"2024-09-12T09:30:43.967238+0000","last_unstale":"2024-09-12T09:31:40.751359+0000","last_undegraded":"2024-09-12T09:31:40.751359+0000","last_fullsized":"2024-09-12T09:31:40.751359+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233371,"stat_sum":{"num_bytes":32613065,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":3713,"num_read_kb":12000,"num_write":383,"num_write_kb":5210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":970649,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[3],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'328","reported_seq":2867,"reported_epoch":126,"state":"activating+undersized+remapped","last_fresh":"2024-09-12T09:31:40.751216+0000","last_change":"2024-09-12T09:31:40.754574+0000","last_active":"2024-09-12T09:31:38.748435+0000","last_peered":"2024-09-12T09:31:38.454192+0000","last_clean":"2024-09-12T09:31:38.454192+0000","last_became_active":"2024-09-12T09:31:12.214331+0000","last_became_peered":"2024-09-12T09:31:12.214331+0000","last_unstale":"2024-09-12T09:31:40.751216+0000","last_undegraded":"2024-09-12T09:31:40.751216+0000","last_fullsized":"2024-09-12T09:31:40.751216+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073810300000000004,"stat_sum":{"num_bytes":18343076,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":1861,"num_read_kb":8773,"num_write":388,"num_write_kb":7855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":4452429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"3","objects":50}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"124'362","reported_seq":3559,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.761145+0000","last_change":"2024-09-12T09:31:12.221774+0000","last_active":"2024-09-12T09:31:40.761145+0000","last_peered":"2024-09-12T09:31:40.761145+0000","last_clean":"2024-09-12T09:31:40.761145+0000","last_became_active":"2024-09-12T09:31:12.220032+0000","last_became_peered":"2024-09-12T09:31:12.220032+0000","last_unstale":"2024-09-12T09:31:40.761145+0000","last_undegraded":"2024-09-12T09:31:40.761145+0000","last_fullsized":"2024-09-12T09:31:40.761145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:34.620230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024686460000000001,"stat_sum":{"num_bytes":55733917,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":12,"num_read":4441,"num_read_kb":51664,"num_write":467,"num_write_kb":39407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"123'383","reported_seq":14219,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.761028+0000","last_change":"2024-09-12T09:30:33.669800+0000","last_active":"2024-09-12T09:31:40.761028+0000","last_peered":"2024-09-12T09:31:40.761028+0000","last_clean":"2024-09-12T09:31:40.761028+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:31:40.761028+0000","last_undegraded":"2024-09-12T09:31:40.761028+0000","last_fullsized":"2024-09-12T09:31:40.761028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:33.296987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017404759999999999,"stat_sum":{"num_bytes":68205903,"num_objects":66,"num_object_clones":19,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":14229,"num_read_kb":71465,"num_write":470,"num_write_kb":29464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"123'648","reported_seq":8031,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784637+0000","last_change":"2024-09-12T09:30:34.672921+0000","last_active":"2024-09-12T09:31:41.784637+0000","last_peered":"2024-09-12T09:31:41.784637+0000","last_clean":"2024-09-12T09:31:41.784637+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:31:41.784637+0000","last_undegraded":"2024-09-12T09:31:41.784637+0000","last_fullsized":"2024-09-12T09:31:41.784637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:08.532133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012203260000000001,"stat_sum":{"num_bytes":107033552,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":8787,"num_read_kb":99424,"num_write":899,"num_write_kb":45203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"123'730","reported_seq":11720,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.783043+0000","last_change":"2024-09-12T09:30:33.668610+0000","last_active":"2024-09-12T09:31:41.783043+0000","last_peered":"2024-09-12T09:31:41.783043+0000","last_clean":"2024-09-12T09:31:41.783043+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:31:41.783043+0000","last_undegraded":"2024-09-12T09:31:41.783043+0000","last_fullsized":"2024-09-12T09:31:41.783043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:36.892794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015375740000000001,"stat_sum":{"num_bytes":50882596,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":18,"num_read":12946,"num_read_kb":51578,"num_write":1069,"num_write_kb":30172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.763490+0000","last_change":"2024-09-12T09:31:21.272112+0000","last_active":"2024-09-12T09:31:40.763490+0000","last_peered":"2024-09-12T09:31:40.763490+0000","last_clean":"2024-09-12T09:31:40.763490+0000","last_became_active":"2024-09-12T09:31:21.271582+0000","last_became_peered":"2024-09-12T09:31:21.271582+0000","last_unstale":"2024-09-12T09:31:40.763490+0000","last_undegraded":"2024-09-12T09:31:40.763490+0000","last_fullsized":"2024-09-12T09:31:40.763490+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:15:48.094875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'240","reported_seq":6119,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-12T09:31:36.760617+0000","last_change":"2024-09-12T09:31:12.230044+0000","last_active":"2024-09-12T09:31:36.760617+0000","last_peered":"2024-09-12T09:31:36.760617+0000","last_clean":"2024-09-12T09:31:36.760617+0000","last_became_active":"2024-09-12T09:31:12.228291+0000","last_became_peered":"2024-09-12T09:31:12.228291+0000","last_unstale":"2024-09-12T09:31:36.760617+0000","last_undegraded":"2024-09-12T09:31:36.760617+0000","last_fullsized":"2024-09-12T09:31:36.760617+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:31.895610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":39319769,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":6337,"num_read_kb":27868,"num_write":230,"num_write_kb":7376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":11634641,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-12T09:31:39.764619+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:31:39.764619+0000","last_peered":"2024-09-12T09:31:39.764619+0000","last_clean":"2024-09-12T09:31:39.764619+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:31:39.764619+0000","last_undegraded":"2024-09-12T09:31:39.764619+0000","last_fullsized":"2024-09-12T09:31:39.764619+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'275","reported_seq":2594,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784514+0000","last_change":"2024-09-12T09:30:33.673235+0000","last_active":"2024-09-12T09:31:41.784514+0000","last_peered":"2024-09-12T09:31:41.784514+0000","last_clean":"2024-09-12T09:31:41.784514+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:31:41.784514+0000","last_undegraded":"2024-09-12T09:31:41.784514+0000","last_fullsized":"2024-09-12T09:31:41.784514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:03.524376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079246000000000004,"stat_sum":{"num_bytes":60066977,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":2172,"num_read_kb":19435,"num_write":332,"num_write_kb":24350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784368+0000","last_change":"2024-09-12T09:31:09.176142+0000","last_active":"2024-09-12T09:31:41.784368+0000","last_peered":"2024-09-12T09:31:41.784368+0000","last_clean":"2024-09-12T09:31:41.784368+0000","last_became_active":"2024-09-12T09:31:09.175686+0000","last_became_peered":"2024-09-12T09:31:09.175686+0000","last_unstale":"2024-09-12T09:31:41.784368+0000","last_undegraded":"2024-09-12T09:31:41.784368+0000","last_fullsized":"2024-09-12T09:31:41.784368+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:50.409345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'862","reported_seq":6593,"reported_epoch":126,"state":"activating+undersized+remapped","last_fresh":"2024-09-12T09:31:40.753995+0000","last_change":"2024-09-12T09:31:40.758161+0000","last_active":"2024-09-12T09:31:39.753013+0000","last_peered":"2024-09-12T09:31:38.456353+0000","last_clean":"2024-09-12T09:31:38.456353+0000","last_became_active":"2024-09-12T09:30:46.983676+0000","last_became_peered":"2024-09-12T09:30:46.983676+0000","last_unstale":"2024-09-12T09:31:40.753995+0000","last_undegraded":"2024-09-12T09:31:40.753995+0000","last_fullsized":"2024-09-12T09:31:40.753995+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069059199999999996,"stat_sum":{"num_bytes":41987485,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":96,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":9425,"num_read_kb":31647,"num_write":1414,"num_write_kb":48045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":7432750,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","5"],"object_location_counts":[{"shards":"2","objects":54}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-12T09:31:39.766467+0000","last_change":"2024-09-12T09:31:39.765592+0000","last_active":"2024-09-12T09:31:39.766467+0000","last_peered":"2024-09-12T09:31:39.766467+0000","last_clean":"2024-09-12T09:31:39.766467+0000","last_became_active":"2024-09-12T09:31:39.765315+0000","last_became_peered":"2024-09-12T09:31:39.765315+0000","last_unstale":"2024-09-12T09:31:39.766467+0000","last_undegraded":"2024-09-12T09:31:39.766467+0000","last_fullsized":"2024-09-12T09:31:39.766467+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:09.749713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'196","reported_seq":13542,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.761344+0000","last_change":"2024-09-12T09:29:36.687876+0000","last_active":"2024-09-12T09:31:40.761344+0000","last_peered":"2024-09-12T09:31:40.761344+0000","last_clean":"2024-09-12T09:31:40.761344+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:31:40.761344+0000","last_undegraded":"2024-09-12T09:31:40.761344+0000","last_fullsized":"2024-09-12T09:31:40.761344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:58.601862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021994889999999998,"stat_sum":{"num_bytes":36511988,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":13393,"num_read_kb":56328,"num_write":175,"num_write_kb":11847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"126'842","reported_seq":5252,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.782987+0000","last_change":"2024-09-12T09:31:24.690964+0000","last_active":"2024-09-12T09:31:41.782987+0000","last_peered":"2024-09-12T09:31:41.782987+0000","last_clean":"2024-09-12T09:31:41.782987+0000","last_became_active":"2024-09-12T09:31:24.688693+0000","last_became_peered":"2024-09-12T09:31:24.688693+0000","last_unstale":"2024-09-12T09:31:41.782987+0000","last_undegraded":"2024-09-12T09:31:41.782987+0000","last_fullsized":"2024-09-12T09:31:41.782987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:30:52.947417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476731,"stat_sum":{"num_bytes":28424307,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":25,"num_read":7477,"num_read_kb":24464,"num_write":1235,"num_write_kb":11083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":5481109,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":272,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.760802+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:31:40.760802+0000","last_peered":"2024-09-12T09:31:40.760802+0000","last_clean":"2024-09-12T09:31:40.760802+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:31:40.760802+0000","last_undegraded":"2024-09-12T09:31:40.760802+0000","last_fullsized":"2024-09-12T09:31:40.760802+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"123'320","reported_seq":4570,"reported_epoch":126,"state":"activating+undersized+remapped","last_fresh":"2024-09-12T09:31:40.752788+0000","last_change":"2024-09-12T09:31:40.761016+0000","last_active":"2024-09-12T09:31:39.750468+0000","last_peered":"2024-09-12T09:31:38.315194+0000","last_clean":"2024-09-12T09:31:38.315194+0000","last_became_active":"2024-09-12T09:31:24.679426+0000","last_became_peered":"2024-09-12T09:31:24.679426+0000","last_unstale":"2024-09-12T09:31:40.752788+0000","last_undegraded":"2024-09-12T09:31:40.752788+0000","last_fullsized":"2024-09-12T09:31:40.752788+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056463300000000004,"stat_sum":{"num_bytes":48133021,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":26,"num_read":4368,"num_read_kb":37750,"num_write":294,"num_write_kb":16406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":16260412,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","6"],"object_location_counts":[{"shards":"2","objects":70}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":231,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.760640+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:31:40.760640+0000","last_peered":"2024-09-12T09:31:40.760640+0000","last_clean":"2024-09-12T09:31:40.760640+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:31:40.760640+0000","last_undegraded":"2024-09-12T09:31:40.760640+0000","last_fullsized":"2024-09-12T09:31:40.760640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"123'318","reported_seq":12438,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.782932+0000","last_change":"2024-09-12T09:31:12.224528+0000","last_active":"2024-09-12T09:31:41.782932+0000","last_peered":"2024-09-12T09:31:41.782932+0000","last_clean":"2024-09-12T09:31:41.782932+0000","last_became_active":"2024-09-12T09:31:12.223598+0000","last_became_peered":"2024-09-12T09:31:12.223598+0000","last_unstale":"2024-09-12T09:31:41.782932+0000","last_undegraded":"2024-09-12T09:31:41.782932+0000","last_fullsized":"2024-09-12T09:31:41.782932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:59:18.043573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083444300000000003,"stat_sum":{"num_bytes":43529263,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":12866,"num_read_kb":38294,"num_write":301,"num_write_kb":24170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":10010383,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784014+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:31:41.784014+0000","last_peered":"2024-09-12T09:31:41.784014+0000","last_clean":"2024-09-12T09:31:41.784014+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:31:41.784014+0000","last_undegraded":"2024-09-12T09:31:41.784014+0000","last_fullsized":"2024-09-12T09:31:41.784014+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"123'275","reported_seq":5013,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.763385+0000","last_change":"2024-09-12T09:31:24.690362+0000","last_active":"2024-09-12T09:31:40.763385+0000","last_peered":"2024-09-12T09:31:40.763385+0000","last_clean":"2024-09-12T09:31:40.763385+0000","last_became_active":"2024-09-12T09:31:24.688009+0000","last_became_peered":"2024-09-12T09:31:24.688009+0000","last_unstale":"2024-09-12T09:31:40.763385+0000","last_undegraded":"2024-09-12T09:31:40.763385+0000","last_fullsized":"2024-09-12T09:31:40.763385+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:42:19.752572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072148700000000004,"stat_sum":{"num_bytes":54130471,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":5731,"num_read_kb":45437,"num_write":287,"num_write_kb":13165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4653123,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":231,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.760921+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:31:40.760921+0000","last_peered":"2024-09-12T09:31:40.760921+0000","last_clean":"2024-09-12T09:31:40.760921+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:31:40.760921+0000","last_undegraded":"2024-09-12T09:31:40.760921+0000","last_fullsized":"2024-09-12T09:31:40.760921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.783803+0000","last_change":"2024-09-12T09:31:21.283339+0000","last_active":"2024-09-12T09:31:41.783803+0000","last_peered":"2024-09-12T09:31:41.783803+0000","last_clean":"2024-09-12T09:31:41.783803+0000","last_became_active":"2024-09-12T09:31:21.277809+0000","last_became_peered":"2024-09-12T09:31:21.277809+0000","last_unstale":"2024-09-12T09:31:41.783803+0000","last_undegraded":"2024-09-12T09:31:41.783803+0000","last_fullsized":"2024-09-12T09:31:41.783803+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:28.759873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"123'1161","reported_seq":5352,"reported_epoch":126,"state":"activating+undersized+remapped","last_fresh":"2024-09-12T09:31:40.750752+0000","last_change":"2024-09-12T09:31:40.768140+0000","last_active":"2024-09-12T09:31:38.749753+0000","last_peered":"2024-09-12T09:31:37.955940+0000","last_clean":"2024-09-12T09:31:37.955940+0000","last_became_active":"2024-09-12T09:30:46.980108+0000","last_became_peered":"2024-09-12T09:30:46.980108+0000","last_unstale":"2024-09-12T09:31:40.750752+0000","last_undegraded":"2024-09-12T09:31:40.750752+0000","last_fullsized":"2024-09-12T09:31:40.750752+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359376,"stat_sum":{"num_bytes":56763747,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":26,"num_read":10552,"num_read_kb":56448,"num_write":1844,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":10746792,"num_keys_recovered":21,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3","objects":62}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"123'593","reported_seq":4375,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.783202+0000","last_change":"2024-09-12T09:29:36.673453+0000","last_active":"2024-09-12T09:31:41.783202+0000","last_peered":"2024-09-12T09:31:41.783202+0000","last_clean":"2024-09-12T09:31:41.783202+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:31:41.783202+0000","last_undegraded":"2024-09-12T09:31:41.783202+0000","last_fullsized":"2024-09-12T09:31:41.783202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:14.604456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012652449999999999,"stat_sum":{"num_bytes":43354714,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":23,"num_read":5669,"num_read_kb":27574,"num_write":810,"num_write_kb":13766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"123'333","reported_seq":5243,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.763507+0000","last_change":"2024-09-12T09:31:23.687201+0000","last_active":"2024-09-12T09:31:40.763507+0000","last_peered":"2024-09-12T09:31:40.763507+0000","last_clean":"2024-09-12T09:31:40.763507+0000","last_became_active":"2024-09-12T09:31:23.683236+0000","last_became_peered":"2024-09-12T09:31:23.683236+0000","last_unstale":"2024-09-12T09:31:40.763507+0000","last_undegraded":"2024-09-12T09:31:40.763507+0000","last_fullsized":"2024-09-12T09:31:40.763507+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:42.047998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094720399999999999,"stat_sum":{"num_bytes":33860656,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":15,"num_read":4894,"num_read_kb":16161,"num_write":421,"num_write_kb":10129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"120'389","reported_seq":6093,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.753351+0000","last_change":"2024-09-12T09:31:12.227245+0000","last_active":"2024-09-12T09:31:40.753351+0000","last_peered":"2024-09-12T09:31:40.753351+0000","last_clean":"2024-09-12T09:31:40.753351+0000","last_became_active":"2024-09-12T09:31:12.226373+0000","last_became_peered":"2024-09-12T09:31:12.226373+0000","last_unstale":"2024-09-12T09:31:40.753351+0000","last_undegraded":"2024-09-12T09:31:40.753351+0000","last_fullsized":"2024-09-12T09:31:40.753351+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:23:26.088804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083970599999999996,"stat_sum":{"num_bytes":76945604,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":6420,"num_read_kb":71929,"num_write":566,"num_write_kb":32702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"123'332","reported_seq":3429,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.762102+0000","last_change":"2024-09-12T09:29:36.686343+0000","last_active":"2024-09-12T09:31:40.762102+0000","last_peered":"2024-09-12T09:31:40.762102+0000","last_clean":"2024-09-12T09:31:40.762102+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:31:40.762102+0000","last_undegraded":"2024-09-12T09:31:40.762102+0000","last_fullsized":"2024-09-12T09:31:40.762102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:19:13.676855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402145,"stat_sum":{"num_bytes":48383587,"num_objects":68,"num_object_clones":23,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":24,"num_read":4675,"num_read_kb":39889,"num_write":349,"num_write_kb":17982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"123'355","reported_seq":8096,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784363+0000","last_change":"2024-09-12T09:30:33.675916+0000","last_active":"2024-09-12T09:31:41.784363+0000","last_peered":"2024-09-12T09:31:41.784363+0000","last_clean":"2024-09-12T09:31:41.784363+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:31:41.784363+0000","last_undegraded":"2024-09-12T09:31:41.784363+0000","last_fullsized":"2024-09-12T09:31:41.784363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:16:47.555515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022846899999999998,"stat_sum":{"num_bytes":77195514,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":8605,"num_read_kb":27014,"num_write":428,"num_write_kb":5809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"126'330","reported_seq":5010,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.785249+0000","last_change":"2024-09-12T09:31:12.226125+0000","last_active":"2024-09-12T09:31:41.785249+0000","last_peered":"2024-09-12T09:31:41.785249+0000","last_clean":"2024-09-12T09:31:41.785249+0000","last_became_active":"2024-09-12T09:31:12.225225+0000","last_became_peered":"2024-09-12T09:31:12.225225+0000","last_unstale":"2024-09-12T09:31:41.785249+0000","last_undegraded":"2024-09-12T09:31:41.785249+0000","last_fullsized":"2024-09-12T09:31:41.785249+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:58:33.489005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":66098273,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":5371,"num_read_kb":92024,"num_write":365,"num_write_kb":45282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":87332658,"num_keys_recovered":59,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"123'512","reported_seq":6658,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784615+0000","last_change":"2024-09-12T09:31:12.222387+0000","last_active":"2024-09-12T09:31:41.784615+0000","last_peered":"2024-09-12T09:31:41.784615+0000","last_clean":"2024-09-12T09:31:41.784615+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:31:41.784615+0000","last_undegraded":"2024-09-12T09:31:41.784615+0000","last_fullsized":"2024-09-12T09:31:41.784615+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:47:56.868393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022377489999999998,"stat_sum":{"num_bytes":97728098,"num_objects":75,"num_object_clones":21,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":24,"num_read":6945,"num_read_kb":79654,"num_write":643,"num_write_kb":45880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"123'437","reported_seq":2831,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-12T09:31:40.763933+0000","last_change":"2024-09-12T09:31:24.691099+0000","last_active":"2024-09-12T09:31:40.763933+0000","last_peered":"2024-09-12T09:31:40.763933+0000","last_clean":"2024-09-12T09:31:40.763933+0000","last_became_active":"2024-09-12T09:31:24.688190+0000","last_became_peered":"2024-09-12T09:31:24.688190+0000","last_unstale":"2024-09-12T09:31:40.763933+0000","last_undegraded":"2024-09-12T09:31:40.763933+0000","last_fullsized":"2024-09-12T09:31:40.763933+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:08:54.657980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078906600000000005,"stat_sum":{"num_bytes":50266684,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":2714,"num_read_kb":21905,"num_write":512,"num_write_kb":24816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":17419997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"123'534","reported_seq":2924,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.783856+0000","last_change":"2024-09-12T09:30:30.987319+0000","last_active":"2024-09-12T09:31:41.783856+0000","last_peered":"2024-09-12T09:31:41.783856+0000","last_clean":"2024-09-12T09:31:41.783856+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:31:41.783856+0000","last_undegraded":"2024-09-12T09:31:41.783856+0000","last_fullsized":"2024-09-12T09:31:41.783856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:31:13.627703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011675019999999999,"stat_sum":{"num_bytes":66278570,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":21,"num_read":5306,"num_read_kb":16745,"num_write":689,"num_write_kb":8213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"124'331","reported_seq":11995,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.783996+0000","last_change":"2024-09-12T09:31:12.224349+0000","last_active":"2024-09-12T09:31:41.783996+0000","last_peered":"2024-09-12T09:31:41.783996+0000","last_clean":"2024-09-12T09:31:41.783996+0000","last_became_active":"2024-09-12T09:31:12.220300+0000","last_became_peered":"2024-09-12T09:31:12.220300+0000","last_unstale":"2024-09-12T09:31:41.783996+0000","last_undegraded":"2024-09-12T09:31:41.783996+0000","last_fullsized":"2024-09-12T09:31:41.783996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:45.678635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":88982912,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":12253,"num_read_kb":104321,"num_write":430,"num_write_kb":56011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":132080671,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"123'196","reported_seq":1405,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-12T09:31:41.784238+0000","last_change":"2024-09-12T09:30:31.088257+0000","last_active":"2024-09-12T09:31:41.784238+0000","last_peered":"2024-09-12T09:31:41.784238+0000","last_clean":"2024-09-12T09:31:41.784238+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:31:41.784238+0000","last_undegraded":"2024-09-12T09:31:41.784238+0000","last_fullsized":"2024-09-12T09:31:41.784238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:33.820586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018037979999999999,"stat_sum":{"num_bytes":37706419,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":1511,"num_read_kb":9201,"num_write":282,"num_write_kb":12299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1729087417,"num_objects":1801,"num_object_clones":477,"num_object_copies":3602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1801,"num_whiteouts":639,"num_read":223812,"num_read_kb":1430570,"num_write":18421,"num_write_kb":790203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2544,"num_bytes_recovered":614104834,"num_keys_recovered":482,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101699584,"data_stored":1341526525,"data_compressed":19890154,"data_compressed_allocated":88961024,"data_compressed_original":1333223790,"omap_allocated":0,"internal_metadata":0},"log_size":14252,"ondisk_log_size":14252,"up":64,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76076,"kb_used_data":22676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295764,"statfs":{"total":96636764160,"available":96558862336,"internally_reserved":0,"allocated":23220224,"data_stored":277357304,"data_compressed":4654469,"data_compressed_allocated":18989056,"data_compressed_original":274773913,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509473,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72732,"kb_used_data":19328,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299108,"statfs":{"total":96636764160,"available":96562286592,"internally_reserved":0,"allocated":19791872,"data_stored":219002532,"data_compressed":4317852,"data_compressed_allocated":15736832,"data_compressed_original":216620939,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41368,"kb_used_data":6020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330472,"statfs":{"total":96636764160,"available":96594403328,"internally_reserved":0,"allocated":6164480,"data_stored":40018594,"data_compressed":1281173,"data_compressed_allocated":3751936,"data_compressed_original":38780396,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":14532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":14880768,"data_stored":184259305,"data_compressed":2283655,"data_compressed_allocated":11718656,"data_compressed_original":182489837,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75956,"kb_used_data":18396,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295884,"statfs":{"total":96636764160,"available":96558985216,"internally_reserved":0,"allocated":18837504,"data_stored":239937392,"data_compressed":3283797,"data_compressed_allocated":15638528,"data_compressed_original":238250211,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60312,"kb_used_data":11076,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311528,"statfs":{"total":96636764160,"available":96575004672,"internally_reserved":0,"allocated":11341824,"data_stored":134906667,"data_compressed":1941894,"data_compressed_allocated":8470528,"data_compressed_original":133282368,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40972,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330868,"statfs":{"total":96636764160,"available":96594808832,"internally_reserved":0,"allocated":1490944,"data_stored":703816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":60129542177,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16519168,"data_stored":251472716,"data_compressed":2140622,"data_compressed_allocated":14729216,"data_compressed_original":250205774,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9854976,"data_stored":134202870,"data_compressed":1941894,"data_compressed_allocated":8470528,"data_compressed_original":133282368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17350656,"data_stored":239233595,"data_compressed":3283797,"data_compressed_allocated":15638528,"data_compressed_original":238250211,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":18280448,"data_stored":218286511,"data_compressed":4317852,"data_compressed_allocated":15736832,"data_compressed_original":216620939,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13389824,"data_stored":183555489,"data_compressed":2283655,"data_compressed_allocated":11718656,"data_compressed_original":182489837,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":38724429,"data_compressed":1274519,"data_compressed_allocated":3715072,"data_compressed_original":38190572,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":21667840,"data_stored":276050915,"data_compressed":4647815,"data_compressed_allocated":18952192,"data_compressed_original":274184089,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:31:44.619 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [1, 4, 5, 0] 2024-09-12T09:31:44.619 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '1', '4', '5', '0'] 2024-09-12T09:31:44.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 1 4 5 0 2024-09-12T09:31:44.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:31:44.708 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:44.707+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:44.747 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:44.762 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:44.783 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone44 from 0xd16701f (overlap 0x12f6cbe) to 0xf2bd64 2024-09-12T09:31:44.808 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:44.806+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:44.873 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:44.871+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:44.878 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-12T09:31:44.897 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25932261600001, 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25927300799999, 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25923926599999, 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.259192228, 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25913438800001, 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.259075301, 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.259041114, 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25900977800001, 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258979737, 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258948492, 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258906723, 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258875257, 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258827717, 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25879467999999, 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25874966000001, 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258717002, 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.258685187, 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25862727400001, 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25858529200001, 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.25854931000001, 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:44.991 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:44.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:44.993 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:44.990+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:45.094 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:45.090+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:45.162 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:45.162 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:45.163 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:45.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:45.220 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:45.218+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:45.293 INFO:teuthology.orchestra.run.smithi007.stdout:5338 read 0xa9a98da thru 0xa9aee7e (0x55a5 bytes) 2024-09-12T09:31:45.295 INFO:teuthology.orchestra.run.smithi007.stdout:5341 write 0x86ed4ab thru 0x86f000e (0x2b64 bytes) 2024-09-12T09:31:45.296 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:45.295+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:45.310 INFO:teuthology.orchestra.run.smithi007.stdout:5342 write 0x3dd90b3 thru 0x3de074b (0x7699 bytes) 2024-09-12T09:31:45.318 INFO:teuthology.orchestra.run.smithi007.stdout:5343 write 0x8899cee thru 0x889c2a6 (0x25b9 bytes) 2024-09-12T09:31:45.327 INFO:teuthology.orchestra.run.smithi007.stdout:5344 write 0x7814670 thru 0x781a820 (0x61b1 bytes) 2024-09-12T09:31:45.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:45.339 INFO:teuthology.orchestra.run.smithi007.stdout:5345 trunc from 0xd16701f to 0xc203134 2024-09-12T09:31:45.354 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:45.355 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:45.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:45.392 INFO:teuthology.orchestra.run.smithi007.stdout:5346 writesame 0xe01af99 thru 0xe02360d data_size 0x1(0x8675 bytes) 2024-09-12T09:31:45.392 INFO:teuthology.orchestra.run.smithi007.stdout:5347 read 0x220bf16 thru 0x2218975 (0xca60 bytes) 2024-09-12T09:31:45.393 INFO:teuthology.orchestra.run.smithi007.stdout:5349 read 0x75b456 thru 0x763518 (0x80c3 bytes) 2024-09-12T09:31:45.393 INFO:teuthology.orchestra.run.smithi007.stdout:5350 write 0x1a0455a thru 0x1a04f0c (0x9b3 bytes) 2024-09-12T09:31:45.419 INFO:teuthology.orchestra.run.smithi007.stdout:5351 trunc from 0xe02360e to 0x5ec5683 2024-09-12T09:31:45.420 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:45.418+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:45.463 INFO:teuthology.orchestra.run.smithi007.stdout:5352 writesame 0x2ec326 thru 0x2f5e71 data_size 0x6(0x9b4c bytes) 2024-09-12T09:31:45.480 INFO:teuthology.orchestra.run.smithi007.stdout:5355 write 0x74b5ab6 thru 0x74c3c4b (0xe196 bytes) 2024-09-12T09:31:45.487 INFO:teuthology.orchestra.run.smithi007.stdout:5356 trunc from 0x74c3c4c to 0x55a500f 2024-09-12T09:31:45.494 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:45.497 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:45.494+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:45.513 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.297300747000001, 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.514 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:31:45.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.295177129000002, 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:31:45.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:31:45.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.295157302, 2024-09-12T09:31:45.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:31:45.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.293062269000004, 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:31:45.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.291027661999998, 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:31:45.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:45.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290988945000002, 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:31:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:31:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290951960999998, 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:31:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:31:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290937577000001, 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:31:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290916586000002, 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:31:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290905115000001, 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.290879629999999, 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:31:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:31:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:31:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:31:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.288692349999998, 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:31:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:31:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:31:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:31:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.288681982, 2024-09-12T09:31:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:31:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:31:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.288656015999997, 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:31:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286964193999999, 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:31:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:31:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:31:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286949638000003, 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:31:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:31:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286929268000002, 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:31:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:31:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:31:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286890640000003, 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:31:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286876116999998, 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:31:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:31:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.286865702, 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:31:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:31:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:45.625 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:45.626 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:45.627 INFO:teuthology.orchestra.run.smithi007.stdout:5357 write 0xc84fc0 thru 0xc90b80 (0xbbc1 bytes) 2024-09-12T09:31:45.627 INFO:teuthology.orchestra.run.smithi007.stdout:5358 punch from 0x539eb6d to 0x53ae748, (0xfbdb bytes) 2024-09-12T09:31:45.628 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:45.628 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:45.628 INFO:teuthology.orchestra.run.smithi007.stdout:5361 clone 46 order 21 su 65536 sc 3 2024-09-12T09:31:45.628 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:45.628 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:45.622+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:45.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:45.629 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:45.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:45.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:45.631 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:45.631 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:45.728 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:45.727+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:45.745 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:45.745 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:45.758 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1f pg_upmap_items mapping to [1->4,5->0] 2024-09-12T09:31:45.802 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:31:45.829 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:45.826+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:45.917 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone45 from 0x55a500f (overlap 0x55a500f) to 0x1121003 2024-09-12T09:31:45.922 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:45.922 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:45.929 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:45.927+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:45.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:46.030 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:46.027+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:46.101 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-12T09:31:46.110 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.110 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:46.153 INFO:teuthology.orchestra.run.smithi007.stdout:5362 read 0x5542939 thru 0x554c7a9 (0x9e71 bytes) 2024-09-12T09:31:46.153 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:46.151+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:46.155 INFO:teuthology.orchestra.run.smithi007.stdout:5364 read 0xa963e1 thru 0xa9e462 (0x8082 bytes) 2024-09-12T09:31:46.157 INFO:teuthology.orchestra.run.smithi007.stdout:5366 write 0x2cd570c thru 0x2cd7b17 (0x240c bytes) 2024-09-12T09:31:46.187 INFO:teuthology.orchestra.run.smithi007.stdout:5367 write 0x63b4ca5 thru 0x63c0d43 (0xc09f bytes) 2024-09-12T09:31:46.195 INFO:teuthology.orchestra.run.smithi007.stdout:5368 punch from 0x5681ab0 to 0x568ecac, (0xd1fc bytes) 2024-09-12T09:31:46.195 INFO:teuthology.orchestra.run.smithi007.stdout:5369 read 0x3cf0222 thru 0x3cfe9d2 (0xe7b1 bytes) 2024-09-12T09:31:46.211 INFO:teuthology.orchestra.run.smithi007.stdout:5371 write 0x832f534 thru 0x8337af9 (0x85c6 bytes) 2024-09-12T09:31:46.232 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:46.230+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:46.243 INFO:teuthology.orchestra.run.smithi007.stdout:5372 write 0xe9cd69b thru 0xe9cefb8 (0x191e bytes) 2024-09-12T09:31:46.249 INFO:teuthology.orchestra.run.smithi007.stdout:5373 write 0x7be71fb thru 0x7bf14f3 (0xa2f9 bytes) 2024-09-12T09:31:46.256 INFO:teuthology.orchestra.run.smithi007.stdout:5374 read 0x1e42df5 thru 0x1e47d05 (0x4f11 bytes) 2024-09-12T09:31:46.256 INFO:teuthology.orchestra.run.smithi007.stdout:5375 read 0xe721240 thru 0xe723ebc (0x2c7d bytes) 2024-09-12T09:31:46.257 INFO:teuthology.orchestra.run.smithi007.stdout:5376 write 0xbde53b3 thru 0xbdeb2ef (0x5f3d bytes) 2024-09-12T09:31:46.264 INFO:teuthology.orchestra.run.smithi007.stdout:5377 writesame 0x25a51fd thru 0x25a55b8 data_size 0x2(0x3bc bytes) 2024-09-12T09:31:46.279 INFO:teuthology.orchestra.run.smithi007.stdout:5378 write 0x7d41a31 thru 0x7d43dbc (0x238c bytes) 2024-09-12T09:31:46.283 INFO:teuthology.orchestra.run.smithi007.stdout:5379 write 0x2f5d3ff thru 0x2f5fab3 (0x26b5 bytes) 2024-09-12T09:31:46.293 INFO:teuthology.orchestra.run.smithi007.stdout:5381 write 0x8a90d0e thru 0x8a9c7d7 (0xbaca bytes) 2024-09-12T09:31:46.301 INFO:teuthology.orchestra.run.smithi007.stdout:5383 write 0xabd6c6 thru 0xac7054 (0x998f bytes) 2024-09-12T09:31:46.309 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.309 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:46.332 INFO:teuthology.orchestra.run.smithi007.stdout:5384 punch from 0x17624fd to 0x1767c19, (0x571c bytes) 2024-09-12T09:31:46.332 INFO:teuthology.orchestra.run.smithi007.stdout:5387 write 0xd794877 thru 0xd7a1c00 (0xd38a bytes) 2024-09-12T09:31:46.333 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:46.331+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:46.340 INFO:teuthology.orchestra.run.smithi007.stdout:5389 write 0x4bb3cdf thru 0x4bb9787 (0x5aa9 bytes) 2024-09-12T09:31:46.350 INFO:teuthology.orchestra.run.smithi007.stdout:5390 read 0x339d636 thru 0x33aabe7 (0xd5b2 bytes) 2024-09-12T09:31:46.351 INFO:teuthology.orchestra.run.smithi007.stdout:5391 write 0x27f38b3 thru 0x27fd330 (0x9a7e bytes) 2024-09-12T09:31:46.368 INFO:teuthology.orchestra.run.smithi007.stdout:5392 trunc from 0xe9cefb9 to 0xad7fc94 2024-09-12T09:31:46.388 INFO:teuthology.orchestra.run.smithi007.stdout:5393 punch from 0x9c4175d to 0x9c47ad6, (0x6379 bytes) 2024-09-12T09:31:46.405 INFO:teuthology.orchestra.run.smithi007.stdout:5394 writesame 0xd447e98 thru 0xd450c00 data_size 0xb(0x8d69 bytes) 2024-09-12T09:31:46.405 INFO:teuthology.orchestra.run.smithi007.stdout:5395 read 0xc54d19e thru 0xc551b53 (0x49b6 bytes) 2024-09-12T09:31:46.407 INFO:teuthology.orchestra.run.smithi007.stdout:5396 trunc from 0xd450c01 to 0xbe9290e 2024-09-12T09:31:46.432 INFO:teuthology.orchestra.run.smithi007.stdout:5397 trunc from 0xbe9290e to 0x3ba5747 2024-09-12T09:31:46.433 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:46.431+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:46.477 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.478 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.482 INFO:teuthology.orchestra.run.smithi007.stdout:5398 read 0x2d8111e thru 0x2d82891 (0x1774 bytes) 2024-09-12T09:31:46.483 INFO:teuthology.orchestra.run.smithi007.stdout:5399 read 0x2f142b5 thru 0x2f17e73 (0x3bbf bytes) 2024-09-12T09:31:46.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:46.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:46.502 INFO:teuthology.orchestra.run.smithi007.stdout:5402 write 0x5fa889c thru 0x5fb1d5e (0x94c3 bytes) 2024-09-12T09:31:46.532 INFO:teuthology.orchestra.run.smithi007.stdout:5403 writesame 0xa227c93 thru 0xa22ce54 data_size 0x46(0x51c2 bytes) 2024-09-12T09:31:46.532 INFO:teuthology.orchestra.run.smithi007.stdout:5404 punch from 0x41deccc to 0x41e5ac6, (0x6dfa bytes) 2024-09-12T09:31:46.533 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:46.531+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:46.546 INFO:teuthology.orchestra.run.smithi007.stdout:5405 write 0xb3f91a7 thru 0xb3fba04 (0x285e bytes) 2024-09-12T09:31:46.556 INFO:teuthology.orchestra.run.smithi007.stdout:5406 trunc from 0xb3fba05 to 0xe2b4abc 2024-09-12T09:31:46.567 INFO:teuthology.orchestra.run.smithi007.stdout:5407 writesame 0x1e2b12a thru 0x1e3a6ee data_size 0x1(0xf5c5 bytes) 2024-09-12T09:31:46.627 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.627 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.633 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:46.631+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:46.634 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:46.634 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:46.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:46.650 INFO:teuthology.orchestra.run.smithi007.stdout:5411 trunc from 0xe2b4abc to 0x73614fb 2024-09-12T09:31:46.673 INFO:teuthology.orchestra.run.smithi007.stdout:5412 punch from 0x89b564 to 0x8a390e, (0x83aa bytes) 2024-09-12T09:31:46.673 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:46.681 INFO:teuthology.orchestra.run.smithi007.stdout:5414 trunc from 0x73614fb to 0x3789d2b 2024-09-12T09:31:46.708 INFO:teuthology.orchestra.run.smithi007.stdout:5415 read 0xe93f8c thru 0xe9f4ff (0xb574 bytes) 2024-09-12T09:31:46.734 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:46.730+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:46.745 INFO:teuthology.orchestra.run.smithi007.stdout:5416 write 0xdeb58f4 thru 0xdebb66c (0x5d79 bytes) 2024-09-12T09:31:46.753 INFO:teuthology.orchestra.run.smithi007.stdout:5418 read 0x4251d87 thru 0x4260489 (0xe703 bytes) 2024-09-12T09:31:46.767 INFO:teuthology.orchestra.run.smithi007.stdout:5419 writesame 0xe48454b thru 0xe48c30a data_size 0x20(0x7dc0 bytes) 2024-09-12T09:31:46.786 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.786 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.787 INFO:teuthology.orchestra.run.smithi007.stdout:5423 trunc from 0xe48c30b to 0x631b111 2024-09-12T09:31:46.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:5424 writesame 0xb4e2096 thru 0xb4efc35 data_size 0x70(0xdba0 bytes) 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:5425 punch from 0x739c14b to 0x73a1054, (0x4f09 bytes) 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:5426 read 0x878ba88 thru 0x878d149 (0x16c2 bytes) 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:5427 read 0x6ec7f18 thru 0x6ed3e9f (0xbf88 bytes) 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:5429 read 0x8549e02 thru 0x8555cdc (0xbedb bytes) 2024-09-12T09:31:46.835 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:46.831+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:46.854 INFO:teuthology.orchestra.run.smithi007.stdout:5430 trunc from 0xb4efc36 to 0x55a0c0 2024-09-12T09:31:46.907 INFO:teuthology.orchestra.run.smithi007.stdout:5431 punch from 0x5506d5 to 0x550781, (0xac bytes) 2024-09-12T09:31:46.907 INFO:teuthology.orchestra.run.smithi007.stdout:5432 read 0x4ce8d1 thru 0x4cfcd9 (0x1409 bytes) 2024-09-12T09:31:46.935 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:46.931+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:46.943 INFO:teuthology.orchestra.run.smithi007.stdout:5433 write 0xea9b36e thru 0xeaa3a0b (0x869e bytes) 2024-09-12T09:31:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:5434 write 0x94d2672 thru 0x94de6f0 (0xc07f bytes) 2024-09-12T09:31:46.959 INFO:teuthology.orchestra.run.smithi007.stdout:5435 write 0xa488dd5 thru 0xa494e5c (0xc088 bytes) 2024-09-12T09:31:46.975 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:46.976 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:46.983 INFO:teuthology.orchestra.run.smithi007.stdout:5436 trunc from 0xeaa3a0c to 0x4abef4d 2024-09-12T09:31:46.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:47.005 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.005 INFO:teuthology.orchestra.run.smithi007.stdout:5438 writesame 0x2eff317 thru 0x2f0374f data_size 0x7(0x4439 bytes) 2024-09-12T09:31:47.024 INFO:teuthology.orchestra.run.smithi007.stdout:5441 write 0x4f52cf3 thru 0x4f57ed9 (0x51e7 bytes) 2024-09-12T09:31:47.036 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:47.034+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:47.049 INFO:teuthology.orchestra.run.smithi007.stdout:5443 write 0x8b45ef7 thru 0x8b4f153 (0x925d bytes) 2024-09-12T09:31:47.054 INFO:teuthology.orchestra.run.smithi007.stdout:5444 punch from 0x2f6247f to 0x2f6335f, (0xee0 bytes) 2024-09-12T09:31:47.054 INFO:teuthology.orchestra.run.smithi007.stdout:5447 read 0x85a53df thru 0x85aabbe (0x57e0 bytes) 2024-09-12T09:31:47.054 INFO:teuthology.orchestra.run.smithi007.stdout:5448 writesame 0x4afc3c1 thru 0x4b07404 data_size 0x3a(0xb044 bytes) 2024-09-12T09:31:47.061 INFO:teuthology.orchestra.run.smithi007.stdout:5449 write 0x54aef34 thru 0x54b4fb7 (0x6084 bytes) 2024-09-12T09:31:47.070 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.082 INFO:teuthology.orchestra.run.smithi007.stdout:5452 trunc from 0x8b4f154 to 0x3bbe802 2024-09-12T09:31:47.105 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.134 INFO:teuthology.orchestra.run.smithi007.stdout:5454 write 0xb92e207 thru 0xb93a6ca (0xc4c4 bytes) 2024-09-12T09:31:47.137 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:47.135+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:47.144 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:47.144 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:47.159 INFO:teuthology.orchestra.run.smithi007.stdout:5455 trunc from 0xb93a6cb to 0x4848ce0 2024-09-12T09:31:47.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:47.184 INFO:teuthology.orchestra.run.smithi007.stdout:5459 trunc from 0x4848ce0 to 0x29b5f96 2024-09-12T09:31:47.204 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.205 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.219 INFO:teuthology.orchestra.run.smithi007.stdout:5468 writesame 0x4b542b5 thru 0x4b58d8e data_size 0x43(0x4ada bytes) 2024-09-12T09:31:47.219 INFO:teuthology.orchestra.run.smithi007.stdout:5470 read 0x2810625 thru 0x2817cb7 (0x7693 bytes) 2024-09-12T09:31:47.233 INFO:teuthology.orchestra.run.smithi007.stdout:5471 write 0x69fdfaa thru 0x6a08c1d (0xac74 bytes) 2024-09-12T09:31:47.238 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:47.235+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:47.253 INFO:teuthology.orchestra.run.smithi007.stdout:5472 writesame 0x7481958 thru 0x74857e7 data_size 0x38(0x3e90 bytes) 2024-09-12T09:31:47.253 INFO:teuthology.orchestra.run.smithi007.stdout:5473 punch from 0x4a3b8c3 to 0x4a4ad69, (0xf4a6 bytes) 2024-09-12T09:31:47.253 INFO:teuthology.orchestra.run.smithi007.stdout:5475 punch from 0x13fd631 to 0x140228a, (0x4c59 bytes) 2024-09-12T09:31:47.254 INFO:teuthology.orchestra.run.smithi007.stdout:5476 read 0x2c03099 thru 0x2c0ba1e (0x8986 bytes) 2024-09-12T09:31:47.254 INFO:teuthology.orchestra.run.smithi007.stdout:5477 punch from 0x6b8bd62 to 0x6b8c551, (0x7ef bytes) 2024-09-12T09:31:47.266 INFO:teuthology.orchestra.run.smithi007.stdout:5479 trunc from 0x74857e8 to 0xece5ece 2024-09-12T09:31:47.297 INFO:teuthology.orchestra.run.smithi007.stdout:5480 trunc from 0xece5ece to 0x2fb8bba 2024-09-12T09:31:47.325 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:47.325 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:47.339 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:47.335+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:47.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:47.352 INFO:teuthology.orchestra.run.smithi007.stdout:5481 write 0xb233c5e thru 0xb2409b2 (0xcd55 bytes) 2024-09-12T09:31:47.361 INFO:teuthology.orchestra.run.smithi007.stdout:5482 write 0x11da00d thru 0x11e3e39 (0x9e2d bytes) 2024-09-12T09:31:47.380 INFO:teuthology.orchestra.run.smithi007.stdout:5483 write 0xb7ef70b thru 0xb7fc7d8 (0xd0ce bytes) 2024-09-12T09:31:47.387 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.388 INFO:teuthology.orchestra.run.smithi007.stdout:5486 trunc from 0xb7fc7d9 to 0xc05416a 2024-09-12T09:31:47.399 INFO:teuthology.orchestra.run.smithi007.stdout:5487 read 0x390326f thru 0x3913153 (0xfee5 bytes) 2024-09-12T09:31:47.399 INFO:teuthology.orchestra.run.smithi007.stdout:5488 write 0x2bdc80c thru 0x2be81f9 (0xb9ee bytes) 2024-09-12T09:31:47.407 INFO:teuthology.orchestra.run.smithi007.stdout:5491 write 0x9f35a65 thru 0x9f39ae5 (0x4081 bytes) 2024-09-12T09:31:47.416 INFO:teuthology.orchestra.run.smithi007.stdout:5494 punch from 0x98818ea to 0x9881a32, (0x148 bytes) 2024-09-12T09:31:47.416 INFO:teuthology.orchestra.run.smithi007.stdout:5495 write 0x6ecaf40 thru 0x6ed28ac (0x796d bytes) 2024-09-12T09:31:47.421 INFO:teuthology.orchestra.run.smithi007.stdout:5496 read 0xa53607f thru 0xa537559 (0x14db bytes) 2024-09-12T09:31:47.422 INFO:teuthology.orchestra.run.smithi007.stdout:5497 write 0x5cd001e thru 0x5cd7727 (0x770a bytes) 2024-09-12T09:31:47.427 INFO:teuthology.orchestra.run.smithi007.stdout:5498 read 0x852fbeb thru 0x8538433 (0x8849 bytes) 2024-09-12T09:31:47.428 INFO:teuthology.orchestra.run.smithi007.stdout:5500 read 0x82df087 thru 0x82e2606 (0x3580 bytes) 2024-09-12T09:31:47.428 INFO:teuthology.orchestra.run.smithi007.stdout:5501 read 0x684ae90 thru 0x684d4de (0x264f bytes) 2024-09-12T09:31:47.428 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.428 INFO:teuthology.orchestra.run.smithi007.stdout:5504 write 0x2825a2d thru 0x282aa7d (0x5051 bytes) 2024-09-12T09:31:47.434 INFO:teuthology.orchestra.run.smithi007.stdout:5505 write 0x24f9907 thru 0x2500cee (0x73e8 bytes) 2024-09-12T09:31:47.439 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:47.435+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:47.440 INFO:teuthology.orchestra.run.smithi007.stdout:5507 write 0xaa3cb0a thru 0xaa3df4e (0x1445 bytes) 2024-09-12T09:31:47.446 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:47.467 INFO:teuthology.orchestra.run.smithi007.stdout:5509 trunc from 0xc05416a to 0x220c3a3 2024-09-12T09:31:47.484 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:47.484 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:47.496 INFO:teuthology.orchestra.run.smithi007.stdout:5510 trunc from 0x220c3a3 to 0x241c26c 2024-09-12T09:31:47.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:47.536 INFO:teuthology.orchestra.run.smithi007.stdout:5511 write 0x84cfb16 thru 0x84d0444 (0x92f bytes) 2024-09-12T09:31:47.540 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:47.534+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:47.544 INFO:teuthology.orchestra.run.smithi007.stdout:5512 punch from 0x5e68533 to 0x5e6af22, (0x29ef bytes) 2024-09-12T09:31:47.544 INFO:teuthology.orchestra.run.smithi007.stdout:5513 read 0x2f055e9 thru 0x2f083d6 (0x2dee bytes) 2024-09-12T09:31:47.544 INFO:teuthology.orchestra.run.smithi007.stdout:5514 punch from 0x4448f5c to 0x4457751, (0xe7f5 bytes) 2024-09-12T09:31:47.544 INFO:teuthology.orchestra.run.smithi007.stdout:5515 clone 47 order 22 su 65536 sc 5 2024-09-12T09:31:47.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:47.640 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:47.639+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:47.662 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:47.662 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:47.677 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:47.742 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:47.739+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:47.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:47.816 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:47.821 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:47.821 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:47.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:31:47.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:47.839 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:47.839 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:47.843 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:47.839+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:47.857 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:47.857 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:47.876 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:47.896 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:47.914 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:47.916 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone46 2024-09-12T09:31:47.934 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:47.934 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:47.943 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:47.943+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:47.964 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:47.964 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:47.964 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:47.964 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:47.964 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:47.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:48.044 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:48.043+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:48.049 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-12T09:31:48.073 INFO:teuthology.orchestra.run.smithi007.stdout:5516 read 0x171c748 thru 0x171dd4d (0x1606 bytes) 2024-09-12T09:31:48.074 INFO:teuthology.orchestra.run.smithi007.stdout:5517 clone 48 order 20 su 16384 sc 3 2024-09-12T09:31:48.112 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:48.112 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:48.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:48.112 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 103.820102285, 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:48.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:31:48.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.360606055999995, 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:31:48.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:31:48.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:31:48.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.123 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.987189104999999, 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:31:48.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:31:48.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:31:48.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.987141858000001, 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.986848781999996, 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.134 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:31:48.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.986286849999999, 2024-09-12T09:31:48.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:31:48.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:31:48.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.986235487000002, 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:31:48.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.985662606999995, 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:31:48.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.985510308000002, 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:31:48.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:31:48.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:31:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:31:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.160 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.160 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:48.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.985153217999994, 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:48.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:31:48.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:31:48.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.168 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.984528248000004, 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.984476912000005, 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:31:48.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:31:48.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.983682122000005, 2024-09-12T09:31:48.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:31:48.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.983607231999997, 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:31:48.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:31:48.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.983554912000002, 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.983506313999996, 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:31:48.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.199 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.983251848999998, 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:31:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:31:48.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.979797559000005, 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:31:48.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.922229012000003, 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:48.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:31:48.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.450957775000006, 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:31:48.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:31:48.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:48.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.228 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:48.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:48.233 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:48.226+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:48.330 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:48.327+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:48.401 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:48.401 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:48.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:48.431 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:48.427+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:48.532 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:48.530+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:48.540 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:48.541 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:48.541 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:48.541 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:48.541 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:48.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:48.633 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:48.631+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:48.722 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.30986301900001, 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:48.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.309767757, 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:48.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.306083133, 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:48.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:48.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:48.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:48.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.304278043, 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:48.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.303731368, 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:48.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.302962925, 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:48.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.302552557, 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:48.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:48.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:48.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.30241338099999, 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:48.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.29463457, 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:48.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.29227055600001, 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:48.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:48.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.281518663, 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:48.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.281322272, 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:48.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.279827852, 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:48.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:48.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 95.579584991000004, 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:48.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 95.579562573000004, 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:48.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 95.579539432000004, 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 95.579514473000003, 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 85.536162958999995, 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.919684916999998, 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.919585627000004, 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.829 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:31:48.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:48.831 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:48.827+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:48.931 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:48.927+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:48.941 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone47 from 0x84d0445 (overlap 0x84d0445) to 0x35201b5 2024-09-12T09:31:48.977 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:48.977 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189047 3.2 3:465d2910:::rbd_object_map.1191915fc47e:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:48.972817+0000", 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0050522559999999998, 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005076896, 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189047 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972817+0000", 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972817+0000", 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972820+0000", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.903e-06 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972847+0000", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7586e-05 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972867+0000", 2024-09-12T09:31:48.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9996999999999998e-05 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.972880+0000", 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3217000000000001e-05 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.974366+0000", 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014856769999999999 2024-09-12T09:31:48.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.974481+0000", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115419 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.975011+0000", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00052947900000000002 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.977072+0000", 2024-09-12T09:31:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002060742 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189048 3.2 3:465d2910:::rbd_object_map.1191915fc47e:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:48.973187+0000", 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046819369999999997, 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047654129999999996, 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:31:48.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189048 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973187+0000", 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973187+0000", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973189+0000", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7969999999999999e-06 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973198+0000", 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4660000000000005e-06 2024-09-12T09:31:48.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973217+0000", 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8964999999999999e-05 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973238+0000", 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0794000000000001e-05 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.975563+0000", 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002325068 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.975644+0000", 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1229000000000004e-05 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.975874+0000", 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022942000000000001 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189049 3.2 3:465d2910:::rbd_object_map.1191915fc47e:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:48.973319+0000", 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0045497530000000001, 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047076260000000003, 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:48.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189049 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973319+0000", 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973319+0000", 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:48.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973320+0000", 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2309999999999999e-06 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973326+0000", 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4859999999999997e-06 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:48.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973337+0000", 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0774000000000001e-05 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.973346+0000", 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0939999999999997e-06 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.976274+0000", 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002927818 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.976368+0000", 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4809999999999995e-05 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:48.976632+0000", 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026358799999999998 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:48.993 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:31:48.993 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 3 2024-09-12T09:31:48.993 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:48.995 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:49.032 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:49.031+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:49.114 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:49.115 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:49.115 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:49.115 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:49.115 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:49.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:49.133 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:49.131+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:49.157 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-12T09:31:49.234 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:49.231+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:49.258 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:49.258 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3547 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 417792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.619223977, 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039043328000000002, 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3547 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636828+0000", 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636829+0000", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636842+0000", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3115000000000001e-05 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636854+0000", 2024-09-12T09:31:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1586000000000001e-05 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636863+0000", 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0499999999999997e-06 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675489+0000", 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038625850000000003 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675577+0000", 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8509000000000002e-05 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675871+0000", 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029391399999999999 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3548 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61917436900001, 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039395010000000001, 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3548 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636877+0000", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636878+0000", 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.139e-06 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636882+0000", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6229999999999999e-06 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636891+0000", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4939999999999992e-06 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636899+0000", 2024-09-12T09:31:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9699999999999999e-06 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.675916+0000", 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039016976000000002 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676001+0000", 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4599999999999996e-05 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676272+0000", 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000271208 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3549 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61914062699999, 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039780982999999999, 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3549 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636911+0000", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636912+0000", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7999999999999995e-07 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636925+0000", 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3787e-05 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636936+0000", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1036000000000001e-05 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636945+0000", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3820000000000004e-06 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676304+0000", 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039358632999999997 2024-09-12T09:31:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676384+0000", 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0859000000000006e-05 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676692+0000", 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030760599999999998 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3550 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.619093589, 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040137883999999999, 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3550 2024-09-12T09:31:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636958+0000", 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636959+0000", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8599999999999997e-07 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636962+0000", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.038e-06 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636972+0000", 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6120000000000005e-06 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.636979+0000", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3719999999999999e-06 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676730+0000", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039751412999999999 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.676820+0000", 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0153000000000002e-05 2024-09-12T09:31:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677096+0000", 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027541000000000002 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3551 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61903574900001, 2024-09-12T09:31:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466199000000001, 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3551 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637016+0000", 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637017+0000", 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637020+0000", 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1410000000000001e-06 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637030+0000", 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.063e-05 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637058+0000", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7274e-05 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677153+0000", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040095299000000001 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677238+0000", 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4705999999999998e-05 2024-09-12T09:31:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677482+0000", 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00024436299999999999 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3552 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61897666199999, 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040834066000000002, 2024-09-12T09:31:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3552 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637075+0000", 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637076+0000", 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9199999999999999e-07 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637080+0000", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7249999999999999e-06 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637090+0000", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0739e-05 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637098+0000", 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6750000000000002e-06 2024-09-12T09:31:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677534+0000", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435943000000002 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677628+0000", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3893000000000004e-05 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677909+0000", 2024-09-12T09:31:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028119899999999999 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3553 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618942475, 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041239400000000002, 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3553 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637109+0000", 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637110+0000", 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5000000000000002e-07 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637113+0000", 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.847e-06 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637122+0000", 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7229999999999999e-06 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637130+0000", 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5310000000000004e-06 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.677962+0000", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040831868 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678057+0000", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5635000000000002e-05 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678349+0000", 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029104599999999999 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3554 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61891113900001, 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041648400000000002, 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3554 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637140+0000", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637141+0000", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.37e-07 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637144+0000", 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7269999999999999e-06 2024-09-12T09:31:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637153+0000", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3389999999999999e-06 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637160+0000", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7780000000000002e-06 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678409+0000", 2024-09-12T09:31:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041248971000000002 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678528+0000", 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118597 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678789+0000", 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000261251 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3555 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618881098, 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042060695000000002, 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3555 2024-09-12T09:31:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637171+0000", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5300000000000003e-07 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637174+0000", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637184+0000", 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.346e-06 2024-09-12T09:31:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637191+0000", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4209999999999998e-06 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678851+0000", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041659859 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.678946+0000", 2024-09-12T09:31:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4767999999999998e-05 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679231+0000", 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028566099999999999 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3556 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618849853, 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042589982999999998, 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3556 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637202+0000", 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2600000000000002e-07 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637214+0000", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1815e-05 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637224+0000", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0064999999999999e-05 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637233+0000", 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1450000000000008e-06 2024-09-12T09:31:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679268+0000", 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042035336 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679350+0000", 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2259e-05 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679792+0000", 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044163699999999999 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3557 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61880808399999, 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042951963000000003, 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3557 2024-09-12T09:31:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637244+0000", 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4e-07 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637247+0000", 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8490000000000002e-06 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637257+0000", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9450000000000005e-06 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637264+0000", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2599999999999999e-06 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679826+0000", 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042561865999999997 2024-09-12T09:31:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.679907+0000", 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1009999999999999e-05 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680195+0000", 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028819300000000001 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3558 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618776618, 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043389456999999999, 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3558 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637275+0000", 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637276+0000", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2500000000000005e-07 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637290+0000", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3856000000000001e-05 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637300+0000", 2024-09-12T09:31:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0132000000000001e-05 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637308+0000", 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9209999999999992e-06 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680246+0000", 2024-09-12T09:31:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042938308000000001 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680366+0000", 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012031699999999999 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680664+0000", 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029819800000000001 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3559 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618729078, 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043765553999999998, 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3559 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637323+0000", 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8000000000000004e-07 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637327+0000", 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637336+0000", 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8109999999999997e-06 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637344+0000", 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9459999999999998e-06 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680697+0000", 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043352913 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.680810+0000", 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011302300000000001 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681088+0000", 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027786100000000002 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3560 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61869604100001, 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044226308999999998, 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3560 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637356+0000", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637357+0000", 2024-09-12T09:31:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637360+0000", 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1709999999999998e-06 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637370+0000", 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9199999999999999e-06 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637387+0000", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7316000000000002e-05 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681160+0000", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043772580999999998 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681253+0000", 2024-09-12T09:31:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3431999999999996e-05 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681582+0000", 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032881300000000001 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3561 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61865102100001, 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044540389, 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3561 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637401+0000", 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-07 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637405+0000", 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.264e-06 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637415+0000", 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0264e-05 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637423+0000", 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681615+0000", 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044192352999999997 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681701+0000", 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5951000000000004e-05 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681941+0000", 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023996999999999999 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3562 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618618363, 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044860850000000001, 2024-09-12T09:31:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3562 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637433+0000", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637434+0000", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637437+0000", 2024-09-12T09:31:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8449999999999999e-06 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637446+0000", 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4550000000000001e-06 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637454+0000", 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.681990+0000", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044536754999999997 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682076+0000", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5883000000000004e-05 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682294+0000", 2024-09-12T09:31:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021785399999999999 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3563 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618586548, 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045124573000000001, 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3563 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637465+0000", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637466+0000", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637469+0000", 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7810000000000001e-06 2024-09-12T09:31:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637478+0000", 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9080000000000007e-06 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637486+0000", 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7780000000000007e-06 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682325+0000", 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044839046 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682381+0000", 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.588e-05 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682590+0000", 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208464 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3564 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.618528635, 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045437999999999999, 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3564 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637523+0000", 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637524+0000", 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9400000000000004e-07 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637529+0000", 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637543+0000", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4177e-05 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637552+0000", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3079999999999997e-06 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682638+0000", 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045086380000000002 2024-09-12T09:31:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682728+0000", 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9563999999999998e-05 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682961+0000", 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023300999999999999 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3565 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61848665300001, 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045717083999999998, 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3565 2024-09-12T09:31:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637565+0000", 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637566+0000", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637569+0000", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637581+0000", 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2275e-05 2024-09-12T09:31:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637589+0000", 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.682993+0000", 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04540367 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683068+0000", 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5376999999999993e-05 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683282+0000", 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021364999999999999 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:3566 3.10 3:0fa4508a:::rbd_data.1191d634a7ff.0000000000000006:5 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-12T09:31:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.61845067100001, 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046039779000000003, 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3566 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637601+0000", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637602+0000", 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5499999999999997e-07 2024-09-12T09:31:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637605+0000", 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637623+0000", 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8185999999999998e-05 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.637631+0000", 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1270000000000003e-06 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683315+0000", 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045683458000000003 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683401+0000", 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6491999999999998e-05 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:45.683641+0000", 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023967199999999999 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.355 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:49.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:49.357 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:49.354+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:49.458 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:49.455+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:49.467 INFO:teuthology.orchestra.run.smithi007.stdout:5518 trunc from 0x84d0445 to 0x189c0e4 2024-09-12T09:31:49.516 INFO:teuthology.orchestra.run.smithi007.stdout:5519 write 0x2d5b67c thru 0x2d6854a (0xcecf bytes) 2024-09-12T09:31:49.528 INFO:teuthology.orchestra.run.smithi007.stdout:5520 trunc from 0x2d6854b to 0x2c56b4a 2024-09-12T09:31:49.535 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:49.535 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:49.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:49.575 INFO:teuthology.orchestra.run.smithi007.stdout:5523 write 0x7f4a72f thru 0x7f548cc (0xa19e bytes) 2024-09-12T09:31:49.582 INFO:teuthology.orchestra.run.smithi007.stdout:5524 write 0x22bbd11 thru 0x22c2f56 (0x7246 bytes) 2024-09-12T09:31:49.591 INFO:teuthology.orchestra.run.smithi007.stdout:5526 write 0x4d9ec8d thru 0x4da3815 (0x4b89 bytes) 2024-09-12T09:31:49.596 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:49.594+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:49.598 INFO:teuthology.orchestra.run.smithi007.stdout:5527 writesame 0x3491425 thru 0x349415d data_size 0x11(0x2d39 bytes) 2024-09-12T09:31:49.605 INFO:teuthology.orchestra.run.smithi007.stdout:5528 write 0x66f7347 thru 0x66ffbfc (0x88b6 bytes) 2024-09-12T09:31:49.614 INFO:teuthology.orchestra.run.smithi007.stdout:5529 read 0x47fa71d thru 0x4806dff (0xc6e3 bytes) 2024-09-12T09:31:49.615 INFO:teuthology.orchestra.run.smithi007.stdout:5530 writesame 0x10f0d55 thru 0x10fbf75 data_size 0x1(0xb221 bytes) 2024-09-12T09:31:49.659 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:49.654+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:49.684 INFO:teuthology.orchestra.run.smithi007.stdout:5534 punch from 0x2a44bbb to 0x2a4602f, (0x1474 bytes) 2024-09-12T09:31:49.689 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:49.689 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:49.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:49.707 INFO:teuthology.orchestra.run.smithi007.stdout:5535 writesame 0xb38a118 thru 0xb38c223 data_size 0x2f(0x210c bytes) 2024-09-12T09:31:49.707 INFO:teuthology.orchestra.run.smithi007.stdout:5539 read 0x559e0cc thru 0x55a8367 (0xa29c bytes) 2024-09-12T09:31:49.707 INFO:teuthology.orchestra.run.smithi007.stdout:5540 write 0x8c18afb thru 0x8c1cf60 (0x4466 bytes) 2024-09-12T09:31:49.727 INFO:teuthology.orchestra.run.smithi007.stdout:5541 write 0xbcfe0bd thru 0xbd0523e (0x7182 bytes) 2024-09-12T09:31:49.738 INFO:teuthology.orchestra.run.smithi007.stdout:5542 write 0x18350aa thru 0x183a06e (0x4fc5 bytes) 2024-09-12T09:31:49.741 INFO:teuthology.orchestra.run.smithi007.stdout:5546 read 0x1b5a159 thru 0x1b5ff67 (0x5e0f bytes) 2024-09-12T09:31:49.742 INFO:teuthology.orchestra.run.smithi007.stdout:5547 flatten 2024-09-12T09:31:49.761 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:49.759+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:49.836 INFO:teuthology.orchestra.run.smithi007.stdout:5548 writesame 0xb40fb74 thru 0xb414784 data_size 0x1(0x4c11 bytes) 2024-09-12T09:31:49.851 INFO:teuthology.orchestra.run.smithi007.stdout:5549 read 0x3bd3f4d thru 0x3bd78bb (0x396f bytes) 2024-09-12T09:31:49.851 INFO:teuthology.orchestra.run.smithi007.stdout:5550 read 0x13ec1a3 thru 0x13ed6b2 (0x1510 bytes) 2024-09-12T09:31:49.851 INFO:teuthology.orchestra.run.smithi007.stdout:5551 write 0x5c9fa20 thru 0x5ca4859 (0x4e3a bytes) 2024-09-12T09:31:49.864 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:49.858+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:49.876 INFO:teuthology.orchestra.run.smithi007.stdout:5552 write 0xe652994 thru 0xe660d77 (0xe3e4 bytes) 2024-09-12T09:31:49.885 INFO:teuthology.orchestra.run.smithi007.stdout:5553 write 0xc2ba thru 0x187a7 (0xc4ee bytes) 2024-09-12T09:31:49.890 INFO:teuthology.orchestra.run.smithi007.stdout:5554 punch from 0x32cf36 to 0x33c7ee, (0xf8b8 bytes) 2024-09-12T09:31:49.890 INFO:teuthology.orchestra.run.smithi007.stdout:5555 read 0x809e1d9 thru 0x809f11d (0xf45 bytes) 2024-09-12T09:31:49.890 INFO:teuthology.orchestra.run.smithi007.stdout:5556 clone 49 order 25 su 262144 sc 5 2024-09-12T09:31:49.897 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.682517763999996, 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:31:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:31:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:31:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.680394145999998, 2024-09-12T09:31:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.680374319000002, 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:31:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.678279285999999, 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:31:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676244679, 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:31:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676205961999997, 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:31:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:31:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676168978, 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:31:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:31:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676154594000003, 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:31:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:31:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676133602999997, 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:31:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676122132000003, 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:31:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.676096647000001, 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:31:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:31:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.673909367, 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:31:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:31:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.673898999000002, 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.673873033, 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:31:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:31:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672181211000002, 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:31:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:31:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672166654999998, 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672146284999997, 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:31:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672107656999998, 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:31:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672093134000001, 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:31:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.672082719000002, 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:31:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:50.006 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:50.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:50.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:50.008 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:50.007+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:50.109 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:50.107+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:50.135 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:50.136 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:50.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:50.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:50.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:50.175 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:50.192 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:50.193 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:50.210 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:50.210 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:50.207+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:50.231 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:50.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:50.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:50.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:50.273 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:50.273 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:50.273 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:50.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:50.294 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:50.311 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:50.306+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:50.412 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:50.411+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:50.425 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:50.425 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:50.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:50.532 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:50.531+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:50.607 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:50.607 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:50.614 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:50.611+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:50.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:50.715 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:50.711+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:50.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:50.830 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:50.830 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:50.830 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-12T09:31:50.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-12T09:31:50.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:31:50.852 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:50.851+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:50.917 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:50.914+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:50.979 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:50.979 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:51.052 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:51.051+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:51.119 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:51.115+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:51.126 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.126 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:51.167 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-12T09:31:51.185 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:31:51.183+0000 7f3454e55640 -1 mon.a@0(leader).osd e134 definitely_dead 0 2024-09-12T09:31:51.221 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:51.218+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:51.308 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.308 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:5557 writesame 0x355b521 thru 0x356822c data_size 0x16(0xcd0c bytes) 2024-09-12T09:31:51.321 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:51.318+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:51.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:5559 read 0x480b365 thru 0x480dd1e (0x29ba bytes) 2024-09-12T09:31:51.367 INFO:teuthology.orchestra.run.smithi007.stdout:5560 trunc from 0xe660d78 to 0x69733f6 2024-09-12T09:31:51.422 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:51.418+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:51.464 INFO:teuthology.orchestra.run.smithi007.stdout:5561 write 0xb34a6ad thru 0xb34d7e6 (0x313a bytes) 2024-09-12T09:31:51.471 INFO:teuthology.orchestra.run.smithi007.stdout:5563 writesame 0xa324dd4 thru 0xa33305a data_size 0x1(0xe287 bytes) 2024-09-12T09:31:51.488 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.488 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:51.524 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:51.518+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:51.539 INFO:teuthology.orchestra.run.smithi007.stdout:5566 write 0x881326a thru 0x88140af (0xe46 bytes) 2024-09-12T09:31:51.543 INFO:teuthology.orchestra.run.smithi007.stdout:5567 writesame 0xac23fb7 thru 0xac2701a data_size 0x26(0x3064 bytes) 2024-09-12T09:31:51.551 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.551 INFO:teuthology.orchestra.run.smithi007.stdout:5569 punch from 0x61d54ad to 0x61ded32, (0x9885 bytes) 2024-09-12T09:31:51.551 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.552 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.552 INFO:teuthology.orchestra.run.smithi007.stdout:5572 write 0x3dc2ba4 thru 0x3dd29c8 (0xfe25 bytes) 2024-09-12T09:31:51.555 INFO:teuthology.orchestra.run.smithi007.stdout:5574 read 0x4214bbf thru 0x421a841 (0x5c83 bytes) 2024-09-12T09:31:51.556 INFO:teuthology.orchestra.run.smithi007.stdout:5575 read 0x30de245 thru 0x30e475d (0x6519 bytes) 2024-09-12T09:31:51.557 INFO:teuthology.orchestra.run.smithi007.stdout:5576 write 0x3bd853d thru 0x3bdd508 (0x4fcc bytes) 2024-09-12T09:31:51.560 INFO:teuthology.orchestra.run.smithi007.stdout:5577 read 0x2c2ec05 thru 0x2c3482f (0x5c2b bytes) 2024-09-12T09:31:51.561 INFO:teuthology.orchestra.run.smithi007.stdout:5578 read 0x50c258e thru 0x50c731b (0x4d8e bytes) 2024-09-12T09:31:51.563 INFO:teuthology.orchestra.run.smithi007.stdout:5580 punch from 0x5cb3ccb to 0x5cc3037, (0xf36c bytes) 2024-09-12T09:31:51.563 INFO:teuthology.orchestra.run.smithi007.stdout:5581 writesame 0x4c35c80 thru 0x4c3e128 data_size 0x1(0x84a9 bytes) 2024-09-12T09:31:51.611 INFO:teuthology.orchestra.run.smithi007.stdout:5583 read 0x77935bf thru 0x77a2f67 (0xf9a9 bytes) 2024-09-12T09:31:51.612 INFO:teuthology.orchestra.run.smithi007.stdout:5584 writesame 0x5a48939 thru 0x5a53ef2 data_size 0x7(0xb5ba bytes) 2024-09-12T09:31:51.621 INFO:teuthology.orchestra.run.smithi007.stdout:5585 read 0x522315d thru 0x5227e7b (0x4d1f bytes) 2024-09-12T09:31:51.623 INFO:teuthology.orchestra.run.smithi007.stdout:5586 read 0x8270f85 thru 0x827a2fd (0x9379 bytes) 2024-09-12T09:31:51.624 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:51.622+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:51.640 INFO:teuthology.orchestra.run.smithi007.stdout:5588 write 0xd67064e thru 0xd67a6de (0xa091 bytes) 2024-09-12T09:31:51.644 INFO:teuthology.orchestra.run.smithi007.stdout:5589 write 0x2720190 thru 0x272e11f (0xdf90 bytes) 2024-09-12T09:31:51.659 INFO:teuthology.orchestra.run.smithi007.stdout:5590 trunc from 0xd67a6df to 0x8314356 2024-09-12T09:31:51.668 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.668 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:51.692 INFO:teuthology.orchestra.run.smithi007.stdout:5591 punch from 0x586a6c9 to 0x5871d6e, (0x76a5 bytes) 2024-09-12T09:31:51.718 INFO:teuthology.orchestra.run.smithi007.stdout:5592 write 0xdc605b5 thru 0xdc6373b (0x3187 bytes) 2024-09-12T09:31:51.725 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:51.722+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:5594 punch from 0x987062b to 0x9873fd1, (0x39a6 bytes) 2024-09-12T09:31:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:5596 write 0x82f00b2 thru 0x82f6abe (0x6a0d bytes) 2024-09-12T09:31:51.731 INFO:teuthology.orchestra.run.smithi007.stdout:5598 write 0x5f3676e thru 0x5f3fb67 (0x93fa bytes) 2024-09-12T09:31:51.734 INFO:teuthology.orchestra.run.smithi007.stdout:5599 read 0xd44a7c2 thru 0xd45a793 (0xffd2 bytes) 2024-09-12T09:31:51.734 INFO:teuthology.orchestra.run.smithi007.stdout:5600 write 0x46f14f0 thru 0x46f403c (0x2b4d bytes) 2024-09-12T09:31:51.744 INFO:teuthology.orchestra.run.smithi007.stdout:5601 trunc from 0xdc6373c to 0xa2ac31e 2024-09-12T09:31:51.759 INFO:teuthology.orchestra.run.smithi007.stdout:5602 writesame 0x587dabe thru 0x58825f1 data_size 0x2(0x4b34 bytes) 2024-09-12T09:31:51.767 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.770 INFO:teuthology.orchestra.run.smithi007.stdout:5604 trunc from 0xa2ac31e to 0x8bcec11 2024-09-12T09:31:51.797 INFO:teuthology.orchestra.run.smithi007.stdout:5605 read 0x489fc7d thru 0x48ac452 (0xc7d6 bytes) 2024-09-12T09:31:51.797 INFO:teuthology.orchestra.run.smithi007.stdout:5606 write 0x208e9b5 thru 0x208f0a0 (0x6ec bytes) 2024-09-12T09:31:51.814 INFO:teuthology.orchestra.run.smithi007.stdout:5607 write 0x92e57c2 thru 0x92e7443 (0x1c82 bytes) 2024-09-12T09:31:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:5609 punch from 0x8c26456 to 0x8c332d3, (0xce7d bytes) 2024-09-12T09:31:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:5614 punch from 0x553fb66 to 0x5543f1c, (0x43b6 bytes) 2024-09-12T09:31:51.823 INFO:teuthology.orchestra.run.smithi007.stdout:5615 trunc from 0x92e7444 to 0xb7ec004 2024-09-12T09:31:51.826 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:51.823+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:51.828 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.828 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.0. 2024-09-12T09:31:51.845 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:31:51.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:51.852 INFO:teuthology.orchestra.run.smithi007.stdout:5616 read 0x14a51ad thru 0x14a69d5 (0x1829 bytes) 2024-09-12T09:31:51.860 INFO:teuthology.orchestra.run.smithi007.stdout:5617 trunc from 0xb7ec004 to 0x5d67a3f 2024-09-12T09:31:51.870 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-12T09:31:51.927 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:51.922+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:51.965 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:51.965 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:51.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:52.028 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:52.026+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:52.110 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:52.111 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:52.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:52.128 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:52.127+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:52.230 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:52.227+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:52.285 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:52.286 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:52.300 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:52.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:52.360 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:52.359+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:52.416 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:52.416 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:52.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:52.432 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:52.427+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:52.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:52.494 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:52.510 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:52.510 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:52.527 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:52.527 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:52.533 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:52.530+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:52.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:52.546 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:52.558 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:52.558 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:52.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:52.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:31:52.586 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:52.586 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:52.604 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:52.664 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:52.663+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:52.735 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:52.730+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:52.744 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:52.744 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:52.758 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:31:52.848 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:52.846+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:52.925 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.514443245, 2024-09-12T09:31:52.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:31:52.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:52.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.51434798299999, 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:31:52.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.51066335900001, 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:31:52.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:31:52.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:31:52.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.508858269, 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:31:52.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:31:52.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:31:52.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.50831159400001, 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:31:52.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.50754315099999, 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:31:52.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:31:52.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:52.960 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.507132783, 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:31:52.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:31:52.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.506993607, 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:31:52.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:31:52.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:31:52.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.499214796, 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:31:52.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:31:52.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.496850782, 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:31:52.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.486098889, 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:31:52.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.485902498, 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:31:52.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.484408078, 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:31:52.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:31:52.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:52.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.784165216999995, 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:31:52.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:31:53.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:31:53.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:53.003 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.784142798999994, 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:31:53.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:31:53.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:53.008 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.784119657999995, 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:31:53.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.784094698999994, 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:31:53.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:53.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:31:53.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.740743184999999, 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:31:53.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:31:53.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:31:53.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.124265143000002, 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:31:53.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:31:53.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 89.124165852999994, 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:31:53.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.036 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:53.038 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:53.035+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:53.138 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:53.134+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:53.209 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:53.209 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.223 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:53.264 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:53.262+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:53.340 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:53.339+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:53.381 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:53.382 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:53.441 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:53.438+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:53.516 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7588707499999998, 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:31:53.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7588199370000002, 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:31:53.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:31:53.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7587832730000001, 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:31:53.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7587239399999999, 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:31:53.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:31:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7586939319999999, 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7586750970000002, 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.543 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:31:53.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:31:53.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7586483790000003, 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:31:53.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7586276129999998, 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:31:53.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:31:53.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7586078980000002, 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:31:53.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:31:53.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7585777020000002, 2024-09-12T09:31:53.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:31:53.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7585471340000001, 2024-09-12T09:31:53.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:31:53.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:31:53.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:31:53.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7585283309999999, 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:31:53.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:31:53.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:31:53.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.758479237, 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:31:53.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:31:53.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:31:53.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:31:53.584 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.758444742, 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:31:53.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.758411964, 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:31:53.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:31:53.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7583768329999998, 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.594 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:31:53.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:31:53.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.758334509, 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:31:53.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:31:53.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7582990020000002, 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:31:53.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:31:53.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7582647319999998, 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:31:53.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.758211395, 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:53.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:31:53.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:31:53.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:53.620 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.622 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:53.622 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:53.619+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:53.722 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:53.719+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:53.793 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:31:53.793 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.809 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:53.813 INFO:teuthology.orchestra.run.smithi007.stdout:5619 punch from 0x1392f20 to 0x139acd7, (0x7db7 bytes) 2024-09-12T09:31:53.813 INFO:teuthology.orchestra.run.smithi007.stdout:5620 punch from 0x571d635 to 0x57246fd, (0x70c8 bytes) 2024-09-12T09:31:53.823 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:53.819+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:53.833 INFO:teuthology.orchestra.run.smithi007.stdout:5622 write 0xb925376 thru 0xb926cf5 (0x1980 bytes) 2024-09-12T09:31:53.839 INFO:teuthology.orchestra.run.smithi007.stdout:5623 read 0x53f97b6 thru 0x5403257 (0x9aa2 bytes) 2024-09-12T09:31:53.840 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.840 INFO:teuthology.orchestra.run.smithi007.stdout:5625 read 0x4b6df47 thru 0x4b6dfa7 (0x61 bytes) 2024-09-12T09:31:53.840 INFO:teuthology.orchestra.run.smithi007.stdout:5626 write 0x3261ac4 thru 0x326c22d (0xa76a bytes) 2024-09-12T09:31:53.844 INFO:teuthology.orchestra.run.smithi007.stdout:5627 read 0x2f8fc00 thru 0x2f9ac54 (0xb055 bytes) 2024-09-12T09:31:53.844 INFO:teuthology.orchestra.run.smithi007.stdout:5628 writesame 0x22c6f03 thru 0x22d0076 data_size 0x6(0x9174 bytes) 2024-09-12T09:31:53.849 INFO:teuthology.orchestra.run.smithi007.stdout:5630 trunc from 0xb926cf6 to 0xb74f5b6 2024-09-12T09:31:53.865 INFO:teuthology.orchestra.run.smithi007.stdout:5631 read 0x2202100 thru 0x220ca90 (0xa991 bytes) 2024-09-12T09:31:53.866 INFO:teuthology.orchestra.run.smithi007.stdout:5632 read 0x149a0a4 thru 0x149b751 (0x16ae bytes) 2024-09-12T09:31:53.867 INFO:teuthology.orchestra.run.smithi007.stdout:5633 write 0x9f928e8 thru 0x9fa0016 (0xd72f bytes) 2024-09-12T09:31:53.870 INFO:teuthology.orchestra.run.smithi007.stdout:5634 write 0x73c87f8 thru 0x73d85c8 (0xfdd1 bytes) 2024-09-12T09:31:53.872 INFO:teuthology.orchestra.run.smithi007.stdout:5635 read 0xd0a28a thru 0xd1a10a (0xfe81 bytes) 2024-09-12T09:31:53.873 INFO:teuthology.orchestra.run.smithi007.stdout:5636 punch from 0x2b9c28a to 0x2ba6aff, (0xa875 bytes) 2024-09-12T09:31:53.873 INFO:teuthology.orchestra.run.smithi007.stdout:5637 writesame 0x436d44c thru 0x4376fa3 data_size 0xc(0x9b58 bytes) 2024-09-12T09:31:53.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:31:53.876 INFO:teuthology.orchestra.run.smithi007.stdout:5639 punch from 0xb6a5418 to 0xb6b29d2, (0xd5ba bytes) 2024-09-12T09:31:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:5641 write 0x47ca66 thru 0x47d52d (0xac8 bytes) 2024-09-12T09:31:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.879 INFO:teuthology.orchestra.run.smithi007.stdout:5645 write 0x1c3843b thru 0x1c47be0 (0xf7a6 bytes) 2024-09-12T09:31:53.881 INFO:teuthology.orchestra.run.smithi007.stdout:5646 read 0x228ca6a thru 0x22967f4 (0x9d8b bytes) 2024-09-12T09:31:53.881 INFO:teuthology.orchestra.run.smithi007.stdout:5647 read 0x9c34c23 thru 0x9c3d967 (0x8d45 bytes) 2024-09-12T09:31:53.882 INFO:teuthology.orchestra.run.smithi007.stdout:5648 read 0x5da5574 thru 0x5dacfd8 (0x7a65 bytes) 2024-09-12T09:31:53.882 INFO:teuthology.orchestra.run.smithi007.stdout:5650 write 0x9ace444 thru 0x9acf557 (0x1114 bytes) 2024-09-12T09:31:53.885 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.885 INFO:teuthology.orchestra.run.smithi007.stdout:5657 read 0x3386b7 thru 0x34038c (0x7cd6 bytes) 2024-09-12T09:31:53.886 INFO:teuthology.orchestra.run.smithi007.stdout:5658 read 0x33b3cac thru 0x33baa89 (0x6dde bytes) 2024-09-12T09:31:53.886 INFO:teuthology.orchestra.run.smithi007.stdout:5659 write 0x4e471c9 thru 0x4e489f1 (0x1829 bytes) 2024-09-12T09:31:53.888 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.888 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.905 INFO:teuthology.orchestra.run.smithi007.stdout:5664 trunc from 0xb74f5b6 to 0x578c094 2024-09-12T09:31:53.924 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:53.922+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:53.929 INFO:teuthology.orchestra.run.smithi007.stdout:5665 read 0x3d54db0 thru 0x3d55fa2 (0x11f3 bytes) 2024-09-12T09:31:53.929 INFO:teuthology.orchestra.run.smithi007.stdout:5666 write 0x2bb67e9 thru 0x2bb7948 (0x1160 bytes) 2024-09-12T09:31:53.931 INFO:teuthology.orchestra.run.smithi007.stdout:5668 read 0x27be2e8 thru 0x27cbbca (0xd8e3 bytes) 2024-09-12T09:31:53.939 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:53.939 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:31:53.939 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:31:53.939 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:31:53.940 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:53.946 INFO:teuthology.orchestra.run.smithi007.stdout:5669 write 0x8b59f51 thru 0x8b5f06e (0x511e bytes) 2024-09-12T09:31:53.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:53.958 INFO:teuthology.orchestra.run.smithi007.stdout:5670 trunc from 0x8b5f06f to 0x39759ad 2024-09-12T09:31:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:5671 read 0xc0653e thru 0xc0f095 (0x8b58 bytes) 2024-09-12T09:31:53.969 INFO:teuthology.orchestra.run.smithi007.stdout:5672 read 0x343c2d3 thru 0x3444e69 (0x8b97 bytes) 2024-09-12T09:31:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:53.970 INFO:teuthology.orchestra.run.smithi007.stdout:5674 clone 50 order 20 su 65536 sc 6 2024-09-12T09:31:54.025 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:54.023+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:54.124 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:31:54.124 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:31:54.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:31:54.124 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:31:54.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.907828723999998, 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:31:54.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.905705105999999, 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:31:54.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:31:54.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.905685278999997, 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:31:54.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.142 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.903590246, 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:54.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:31:54.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901555639000001, 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:31:54.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:31:54.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901516921999999, 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:31:54.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.160 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901479938000001, 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.161 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:31:54.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:31:54.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:54.166 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901465553999998, 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:54.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:31:54.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901444562999998, 2024-09-12T09:31:54.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:54.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:31:54.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:31:54.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901433091999998, 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:31:54.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:31:54.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.901407607000003, 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.189 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.899220327000002, 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:31:54.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:31:54.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.899209958999997, 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:31:54.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.899183993000001, 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.206 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897492171000003, 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:31:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897477615, 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:31:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:31:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897457244999998, 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:31:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:31:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897418617, 2024-09-12T09:31:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:31:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:31:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897404094000002, 2024-09-12T09:31:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:31:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:31:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.897393678999997, 2024-09-12T09:31:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:31:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:31:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:31:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:31:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:31:54.242 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:31:54.244 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:54.238+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:54.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:31:54.344 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:54.343+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:54.391 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:54.391 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:54.391 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:31:54.391 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:54.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-12T09:31:54.445 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:54.442+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:54.547 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:54.543+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:54.589 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:31:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:31:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:31:54.589 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:54.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-12T09:31:54.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:31:54.648 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:54.646+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:31:54.760 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:54.759+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:54.789 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:16694 3.12 3:4bce370e:::rbd_object_map.11914330b43e.0000000000000009:head [writefull 0~130] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.65144448300001, 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0033518749999999, 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16694 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137184+0000", 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137187+0000", 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1609999999999999e-06 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137200+0000", 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3066e-05 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137226+0000", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5933000000000001e-05 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137241+0000", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4892000000000001e-05 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137329+0000", 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8084999999999994e-05 2024-09-12T09:31:54.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:54.137340+0000", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0984e-05 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.137942+0000", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0006024950000001 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138102+0000", 2024-09-12T09:31:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015944900000000001 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.138269+0000", 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000167713 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.139429+0000", 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011593720000000001 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:54.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140409+0000", 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098033800000000009 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140443+0000", 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4369999999999998e-05 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:55.140535+0000", 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2016999999999997e-05 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41334 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.550108941, 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094844095000000003, 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41334 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238519+0000", 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238520+0000", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238522+0000", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.65e-06 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238531+0000", 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3450000000000006e-06 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238536+0000", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333168+0000", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094632215000000006 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333215+0000", 2024-09-12T09:31:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6360000000000003e-05 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333363+0000", 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014853699999999999 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41342 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1904640~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.550083489, 2024-09-12T09:31:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095184779999999997, 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41342 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238545+0000", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-07 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238547+0000", 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.306e-06 2024-09-12T09:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238556+0000", 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238561+0000", 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4160000000000003e-06 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333413+0000", 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094851991999999996 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333507+0000", 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3834999999999999e-05 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333729+0000", 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022235899999999999 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41345 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1929216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.55005863700001, 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095520286999999995, 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41345 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238569+0000", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238570+0000", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238572+0000", 2024-09-12T09:31:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.086e-06 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238580+0000", 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.554e-06 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238587+0000", 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2649999999999999e-06 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333779+0000", 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095192449999999998 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.333871+0000", 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1638999999999995e-05 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334090+0000", 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021880900000000001 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41346 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1953792~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.55003329900001, 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095821997000000006, 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41346 2024-09-12T09:31:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238595+0000", 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238602+0000", 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.844e-06 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238611+0000", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238617+0000", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6339999999999998e-06 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334124+0000", 2024-09-12T09:31:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095507155999999996 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334214+0000", 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9920999999999995e-05 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334417+0000", 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020301799999999999 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41347 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1908736~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.550002269, 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096151168999999995, 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41347 2024-09-12T09:31:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238626+0000", 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9500000000000003e-07 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238629+0000", 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238637+0000", 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.507e-06 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238643+0000", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7030000000000003e-06 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334452+0000", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.095808857999999997 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334532+0000", 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0012000000000006e-05 2024-09-12T09:31:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334777+0000", 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000245362 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41348 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.54997710799999, 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096416447000000002, 2024-09-12T09:31:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41348 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238651+0000", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238654+0000", 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3020000000000002e-06 2024-09-12T09:31:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238663+0000", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0059999999999998e-06 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238668+0000", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334801+0000", 2024-09-12T09:31:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096132334 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.334872+0000", 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1182999999999999e-05 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335067+0000", 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019562200000000001 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41349 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1933312~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549951208, 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096722618999999996, 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41349 2024-09-12T09:31:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238677+0000", 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238680+0000", 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238687+0000", 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3529999999999997e-06 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238693+0000", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335110+0000", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096417853999999997 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335183+0000", 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2787000000000006e-05 2024-09-12T09:31:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335400+0000", 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021629799999999999 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41350 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1957888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549927602, 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097008445999999998, 2024-09-12T09:31:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41350 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238700+0000", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238701+0000", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238703+0000", 2024-09-12T09:31:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0260000000000002e-06 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238711+0000", 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1100000000000003e-06 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238717+0000", 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8050000000000003e-06 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335429+0000", 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.096712292000000005 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335508+0000", 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9297999999999999e-05 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335709+0000", 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200464 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41351 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1916928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.54990309, 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097324421999999994, 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41351 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238725+0000", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238728+0000", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2570000000000002e-06 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238736+0000", 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4109999999999996e-06 2024-09-12T09:31:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238742+0000", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335755+0000", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097013109 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.335839+0000", 2024-09-12T09:31:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4598000000000006e-05 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336049+0000", 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000210134 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41352 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549878652, 2024-09-12T09:31:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097597491999999994, 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41352 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238749+0000", 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238750+0000", 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238752+0000", 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2419999999999999e-06 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238761+0000", 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4120000000000001e-06 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238766+0000", 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5369999999999997e-06 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336079+0000", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097313101999999999 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336160+0000", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0393000000000001e-05 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336347+0000", 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000187336 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41353 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.54985406500001, 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098686329000000003, 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41353 2024-09-12T09:31:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238774+0000", 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238775+0000", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238777+0000", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238785+0000", 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4370000000000007e-06 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238790+0000", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.259e-06 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336602+0000", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097811690000000007 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.336697+0000", 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4778999999999995e-05 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337460+0000", 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00076346400000000005 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41356 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1941504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.54983004899999, 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099039434999999995, 2024-09-12T09:31:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41356 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238798+0000", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238801+0000", 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3939999999999999e-06 2024-09-12T09:31:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238812+0000", 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1154e-05 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238818+0000", 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337498+0000", 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098679974000000004 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337590+0000", 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1935000000000007e-05 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337837+0000", 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024792199999999998 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41362 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549802207, 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099372035999999997, 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41362 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238826+0000", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238829+0000", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3769999999999999e-06 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238837+0000", 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3950000000000001e-06 2024-09-12T09:31:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238843+0000", 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337874+0000", 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099031061000000004 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.337960+0000", 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6252e-05 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338198+0000", 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023795700000000001 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41363 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1937408~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549777362, 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099710024999999994, 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41363 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238851+0000", 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.06e-07 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238854+0000", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238862+0000", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4670000000000004e-06 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238867+0000", 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338230+0000", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099363087000000003 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338318+0000", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7902e-05 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338561+0000", 2024-09-12T09:31:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024240500000000001 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41364 3.12 3:4a177fa6:::rbd_data.11913fee8ce1.0000000000000000:8 [read 1961984~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.54975263599999, 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10003258700000001, 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41364 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238875+0000", 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238876+0000", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238878+0000", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238886+0000", 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1079999999999993e-06 2024-09-12T09:31:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238892+0000", 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338612+0000", 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099720119999999995 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338705+0000", 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3450999999999997e-05 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338908+0000", 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020254999999999999 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41365 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1921024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.549728139, 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.100314873, 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41365 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.238900+0000", 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244613+0000", 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057129570000000003 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244652+0000", 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8843000000000001e-05 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244667+0000", 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5274e-05 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244679+0000", 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2163e-05 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338937+0000", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094257933000000002 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.338988+0000", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0608000000000001e-05 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339215+0000", 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022709499999999999 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:41366 3.12 3:4e18b6e8:::rbd_data.11913fee8ce1.0000000000000001:8 [read 1945600~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e45)", 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.543922268, 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094740035, 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 41366 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244706+0000", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2600000000000002e-07 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244710+0000", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244718+0000", 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.32e-06 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.244725+0000", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4800000000000004e-06 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339240+0000", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.094514852999999996 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339297+0000", 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7031000000000003e-05 2024-09-12T09:31:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:29:59.339446+0000", 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014865399999999999 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:49866 3.12 3:4969c675:::rbd_data.119172280d2a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.398970304, 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.11992394100000001, 2024-09-12T09:31:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49866 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389658+0000", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.389660+0000", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8470000000000001e-06 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421873+0000", 2024-09-12T09:31:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032213838000000002 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421921+0000", 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7234999999999998e-05 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.421941+0000", 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0489999999999999e-05 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422112+0000", 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017092100000000001 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.422359+0000", 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024660900000000001 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 7", 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.423233+0000", 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087461199999999998 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.502182+0000", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078949020999999994 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509353+0000", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071710580000000001 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509465+0000", 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112061 2024-09-12T09:31:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:04.509582+0000", 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116249 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116936 3.7 3:e187e6fa:::rbd_data.11916e3c78cf.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 91.581473505999995, 2024-09-12T09:31:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.108046427, 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116936 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207155+0000", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.207156+0000", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.119e-06 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253855+0000", 2024-09-12T09:31:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046699686999999997 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253930+0000", 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4807000000000004e-05 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.253963+0000", 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3036e-05 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:31:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254060+0000", 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6531000000000001e-05 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.254255+0000", 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019499599999999999 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.255512+0000", 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012574229999999999 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.314139+0000", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058626768000000003 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315130+0000", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099125899999999998 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315164+0000", 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3868000000000002e-05 2024-09-12T09:31:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.315201+0000", 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6933e-05 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:31:54.886 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:31:54.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:54.888 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-12T09:31:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-12T09:31:54.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:31:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:31:54.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:31:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:31:54.890 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:31:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:31:54.890 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:31:54.891 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:31:54.891 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:54.887+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:54.905 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:31:54.906 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:31:54.928 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:31:54.928 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:31:54.951 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:31:54.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:31:54.988 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:54.986+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:55.046 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.046 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:55.089 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:55.086+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:55.190 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:55.186+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:55.214 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.215 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:55.291 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:55.286+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:55.363 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.364 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:55.392 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:55.390+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:55.493 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:55.491+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:55.510 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.510 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:55.616 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:55.615+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:55.697 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:55.694+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:55.812 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:55.811+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone49 from 0x39759ad (overlap 0x39759ad) to 0x113ce1a 2024-09-12T09:31:55.898 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:55.895+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:55.914 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:55.914 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:55.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:55.970 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-12T09:31:56.016 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:56.015+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:56.099 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:56.094+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:56.104 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.104 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:56.212 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:56.211+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:56.301 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:56.299+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:56.305 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.305 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:56.403 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:56.402+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:56.503 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:56.503+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:56.514 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.514 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.531 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:56.604 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:31:56.603+0000 7f3f156cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26385) UID: 0 2024-09-12T09:31:56.686 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.686 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.702 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:56.707 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:56.703+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:56.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:31:56.807 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:56.803+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:56.825 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.826 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:56.871 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:56.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:31:56.871 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:31:56.871 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T09:31:56.872 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:31:56.908 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:56.906+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:56.930 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T09:31:56.930 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-12T09:31:56.931 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:31:56.931 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-12T09:31:56.935 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-12T09:31:56.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-12T09:31:56.957 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:56.957 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:56.971 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:31:56.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:31:57.028 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:57.027+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:57.084 INFO:teuthology.orchestra.run.smithi007.stdout:5675 writesame 0xecdb9f0 thru 0xece96a1 data_size 0x29(0xdcb2 bytes) 2024-09-12T09:31:57.084 INFO:teuthology.orchestra.run.smithi007.stdout:5676 punch from 0x8ecf2b6 to 0x8ed8318, (0x9062 bytes) 2024-09-12T09:31:57.084 INFO:teuthology.orchestra.run.smithi007.stdout:5679 clone 51 order 22 su 16384 sc 2 2024-09-12T09:31:57.109 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:57.106+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:57.137 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.137 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.151 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:31:57.211 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:57.207+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:57.312 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:57.311+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:57.312 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.312 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.326 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:31:57.413 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:57.411+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:57.443 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.443 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:3.e 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:1.0 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:2.4 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:2.5 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:3.10 2024-09-12T09:31:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:3.12 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.17 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.3 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.18 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.b 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.1 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.1b 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.1f 2024-09-12T09:31:57.505 INFO:teuthology.orchestra.run.smithi007.stdout:3.6 2024-09-12T09:31:57.513 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:57.510+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:57.576 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.576 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.590 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:31:57.614 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:57.611+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:31:57.715 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:57.710+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:57.763 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.763 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:31:57.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.18 --file /var/log/ceph/exp.3.18.0 2024-09-12T09:31:57.820 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:57.818+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:57.917 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:57.914+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:57.943 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:57.944 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:57.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:31:58.018 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:58.015+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:58.093 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:58.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:31:58.152 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:58.151+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:58.220 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:58.219+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:58.278 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:58.278 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:58.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:31:58.336 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:58.335+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:Exporting 3.18 info 3.18( v 133'378 (0'0,133'378] local-lis/les=0/0 n=61 ec=23/23 lis/c=136/128 les/c/f=137/129/0 sis=140) 2024-09-12T09:31:58.349 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:182d8453:::rbd_data.1191f0cca7c3.000000000000001b:head# 2024-09-12T09:31:58.349 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:188bf538:::rbd_data.11914e57135.0000000000000028:head# 2024-09-12T09:31:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:188da08b:::rbd_object_map.11912e4e65e9.0000000000000006:head# 2024-09-12T09:31:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:18910b52:::rbd_data.1191e0d4a936.000000000000000b:head# 2024-09-12T09:31:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1891e83f:::rbd_data.11914e57135.000000000000004f:head# 2024-09-12T09:31:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:18d14836:::rbd_data.1191b7249abe.000000000000016d:20# 2024-09-12T09:31:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:18d14836:::rbd_data.1191b7249abe.000000000000016d:head# 2024-09-12T09:31:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-12T09:31:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:194660e1:::rbd_data.1191f7cb0ecd.0000000000000001:head# 2024-09-12T09:31:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19bc2ff7:::rbd_data.11918dbccc64.0000000000000004:f# 2024-09-12T09:31:58.354 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19bc2ff7:::rbd_data.11918dbccc64.0000000000000004:head# 2024-09-12T09:31:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19c03697:::rbd_data.11914e57135.0000000000000010:b# 2024-09-12T09:31:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19c03697:::rbd_data.11914e57135.0000000000000010:head# 2024-09-12T09:31:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19cfe708:::rbd_data.1191fc712d4f.000000000000000a:head# 2024-09-12T09:31:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:19fab269:::rbd_header.11914f8d16a3:head# 2024-09-12T09:31:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a1aabe5:::rbd_data.1191f7cb0ecd.0000000000000007:head# 2024-09-12T09:31:58.357 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a57b117:::rbd_data.11914330b43e.000000000000001c:9# 2024-09-12T09:31:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a57b117:::rbd_data.11914330b43e.000000000000001c:head# 2024-09-12T09:31:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a5cda46:::rbd_data.11914330b43e.0000000000000023:head# 2024-09-12T09:31:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a633d33:::rbd_data.11917d6cdfd3.0000000000000019:1c# 2024-09-12T09:31:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1a633d33:::rbd_data.11917d6cdfd3.0000000000000019:head# 2024-09-12T09:31:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1b8a554d:::rbd_data.1191d92ee0a3.0000000000000022:head# 2024-09-12T09:31:58.360 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c139081:::rbd_data.1191b7249abe.000000000000016c:20# 2024-09-12T09:31:58.360 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c139081:::rbd_data.1191b7249abe.000000000000016c:head# 2024-09-12T09:31:58.360 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c43210e:::rbd_object_map.1191410774a4:head# 2024-09-12T09:31:58.360 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c45e6b1:::rbd_data.1191d92ee0a3.0000000000000068:head# 2024-09-12T09:31:58.361 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c7beb88:::rbd_data.1191ea5030de.0000000000000073:1a# 2024-09-12T09:31:58.361 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1c7beb88:::rbd_data.1191ea5030de.0000000000000073:head# 2024-09-12T09:31:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1cbfaa64:::rbd_data.1191b7249abe.0000000000000006:head# 2024-09-12T09:31:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1cc161c0:::rbd_data.119172280d2a.0000000000000007:head# 2024-09-12T09:31:58.363 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1cf74962:::rbd_data.11914da40b12.0000000000000017:16# 2024-09-12T09:31:58.363 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1cf74962:::rbd_data.11914da40b12.0000000000000017:head# 2024-09-12T09:31:58.363 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d31213f:::rbd_data.11912e4e65e9.000000000000005e:head# 2024-09-12T09:31:58.363 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d4f728d:::rbd_data.11914f8d16a3.0000000000000009:30# 2024-09-12T09:31:58.364 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d4f728d:::rbd_data.11914f8d16a3.0000000000000009:head# 2024-09-12T09:31:58.365 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d5d3248:::rbd_data.11914e57135.0000000000000003:b# 2024-09-12T09:31:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d5d3248:::rbd_data.11914e57135.0000000000000003:head# 2024-09-12T09:31:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d6ddbf2:::rbd_data.1191ec363dbc.0000000000000003:head# 2024-09-12T09:31:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1d7fd0d3:::rbd_data.11914330b43e.000000000000004c:head# 2024-09-12T09:31:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1da5b711:::rbd_data.119195859761.0000000000000002:1d# 2024-09-12T09:31:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1da5b711:::rbd_data.119195859761.0000000000000002:head# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1db9b28f:::rbd_data.11914e57135.000000000000004c:head# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1dcd4cda:::rbd_data.1191b7249abe.000000000000005f:20# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1dcd4cda:::rbd_data.1191b7249abe.000000000000005f:head# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1de54f48:::rbd_data.11914e57135.000000000000003a:head# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1ded4b08:::rbd_data.1191c1f77e6b.0000000000000042:head# 2024-09-12T09:31:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e07f715:::rbd_data.11914330b43e.000000000000001d:9# 2024-09-12T09:31:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e07f715:::rbd_data.11914330b43e.000000000000001d:head# 2024-09-12T09:31:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e1cc104:::rbd_data.11914da40b12.0000000000000002:16# 2024-09-12T09:31:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e1cc104:::rbd_data.11914da40b12.0000000000000002:head# 2024-09-12T09:31:58.375 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e78e1b0:::rbd_data.119154ef11c8.0000000000000002:2f# 2024-09-12T09:31:58.381 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1e78e1b0:::rbd_data.119154ef11c8.0000000000000002:head# 2024-09-12T09:31:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1ec02b2f:::rbd_data.1191915fc47e.0000000000000018:head# 2024-09-12T09:31:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1ed43b80:::rbd_data.11914e57135.000000000000002a:head# 2024-09-12T09:31:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1f37ac9b:::rbd_data.11913fee8ce1.0000000000000010:head# 2024-09-12T09:31:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1f46b0e2:::rbd_data.1191f0cca7c3.0000000000000025:13# 2024-09-12T09:31:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1f46b0e2:::rbd_data.1191f0cca7c3.0000000000000025:head# 2024-09-12T09:31:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1f942b46:::rbd_data.11914f8d16a3.0000000000000011:30# 2024-09-12T09:31:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1f942b46:::rbd_data.11914f8d16a3.0000000000000011:head# 2024-09-12T09:31:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1feb8588:::rbd_id.image_client.0-clone41:head# 2024-09-12T09:31:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:Read #3:1ff21f0c:::rbd_object_map.1191e31dd75:head# 2024-09-12T09:31:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:Export successful 2024-09-12T09:31:58.422 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:58.418+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:58.455 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:58.456 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:58.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:31:58.560 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:31:58.559+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:31:58.623 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:58.622+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:58.634 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:58.635 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:58.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:31:58.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.18 2024-09-12T09:31:58.725 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:58.722+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:58.817 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:58.817 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:58.825 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:58.823+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:58.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:31:58.927 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:58.922+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:31:59.027 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:31:59.022+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:31:59.128 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:59.126+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:59.146 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.146 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:31:59.229 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:59.226+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:59.287 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.287 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:31:59.330 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:59.326+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:59.420 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.420 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.431 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:31:59.426+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:31:59.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:31:59.532 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:31:59.531+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:31:59.568 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.568 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:31:59.633 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:59.630+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:31:59.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:31:59.708 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.708 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:31:59.733 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone50 from 0xece96a2 (overlap 0x39759ad) to 0x1cbacd6 2024-09-12T09:31:59.770 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:31:59.766+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:31:59.834 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:31:59.831+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:31:59.900 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-12T09:31:59.905 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:31:59.905 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:31:59.923 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:31:59.947 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:31:59.966 INFO:teuthology.orchestra.run.smithi007.stdout:5684 trunc from 0xece96a2 to 0x197b45b 2024-09-12T09:31:59.968 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:31:59.966+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:00.035 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:00.031+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:00.066 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.066 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.076 INFO:teuthology.orchestra.run.smithi007.stdout:5685 writesame 0x82c322c thru 0x82c7119 data_size 0x3(0x3eee bytes) 2024-09-12T09:32:00.076 INFO:teuthology.orchestra.run.smithi007.stdout:5686 punch from 0x2852a4e to 0x2853507, (0xab9 bytes) 2024-09-12T09:32:00.076 INFO:teuthology.orchestra.run.smithi007.stdout:5687 read 0x32dd086 thru 0x32e7f13 (0xae8e bytes) 2024-09-12T09:32:00.076 INFO:teuthology.orchestra.run.smithi007.stdout:5688 write 0x16fe201 thru 0x1700ae3 (0x28e3 bytes) 2024-09-12T09:32:00.084 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:00.131 INFO:teuthology.orchestra.run.smithi007.stdout:5689 read 0x192532b thru 0x192dc70 (0x8946 bytes) 2024-09-12T09:32:00.132 INFO:teuthology.orchestra.run.smithi007.stdout:5690 punch from 0x148ccda to 0x148e6b9, (0x19df bytes) 2024-09-12T09:32:00.132 INFO:teuthology.orchestra.run.smithi007.stdout:5692 read 0x483e9a9 thru 0x484b90c (0xcf64 bytes) 2024-09-12T09:32:00.132 INFO:teuthology.orchestra.run.smithi007.stdout:5693 punch from 0x60f9aef to 0x610342a, (0x993b bytes) 2024-09-12T09:32:00.132 INFO:teuthology.orchestra.run.smithi007.stdout:5694 write 0x6c38a98 thru 0x6c40560 (0x7ac9 bytes) 2024-09-12T09:32:00.143 INFO:teuthology.orchestra.run.smithi007.stdout:5695 trunc from 0x82c711a to 0x4d6169e 2024-09-12T09:32:00.157 INFO:teuthology.orchestra.run.smithi007.stdout:5696 trunc from 0x4d6169e to 0x51f0a48 2024-09-12T09:32:00.172 INFO:teuthology.orchestra.run.smithi007.stdout:5697 trunc from 0x51f0a48 to 0x12d6f88 2024-09-12T09:32:00.172 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:00.170+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:00.188 INFO:teuthology.orchestra.run.smithi007.stdout:5698 write 0x353d7c thru 0x35695d (0x2be2 bytes) 2024-09-12T09:32:00.237 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:00.235+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:00.267 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.267 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:00.286 INFO:teuthology.orchestra.run.smithi007.stdout:5699 write 0xaeae31f thru 0xaeb4258 (0x5f3a bytes) 2024-09-12T09:32:00.293 INFO:teuthology.orchestra.run.smithi007.stdout:5702 write 0xa0cf8bb thru 0xa0dd011 (0xd757 bytes) 2024-09-12T09:32:00.299 INFO:teuthology.orchestra.run.smithi007.stdout:5703 write 0x84dc953 thru 0x84e9cdb (0xd389 bytes) 2024-09-12T09:32:00.307 INFO:teuthology.orchestra.run.smithi007.stdout:5704 read 0x5e5dbe1 thru 0x5e6af13 (0xd333 bytes) 2024-09-12T09:32:00.307 INFO:teuthology.orchestra.run.smithi007.stdout:5705 write 0x4d10e8c thru 0x4d1503d (0x41b2 bytes) 2024-09-12T09:32:00.315 INFO:teuthology.orchestra.run.smithi007.stdout:5706 read 0x5e79c3f thru 0x5e7eb90 (0x4f52 bytes) 2024-09-12T09:32:00.315 INFO:teuthology.orchestra.run.smithi007.stdout:5707 write 0x9012bcd thru 0x90140f5 (0x1529 bytes) 2024-09-12T09:32:00.321 INFO:teuthology.orchestra.run.smithi007.stdout:5710 write 0x33943a0 thru 0x339d8ef (0x9550 bytes) 2024-09-12T09:32:00.329 INFO:teuthology.orchestra.run.smithi007.stdout:5711 read 0x61c3923 thru 0x61c71b2 (0x3890 bytes) 2024-09-12T09:32:00.338 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:00.335+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:00.339 INFO:teuthology.orchestra.run.smithi007.stdout:5712 write 0xdb2e926 thru 0xdb3bc70 (0xd34b bytes) 2024-09-12T09:32:00.345 INFO:teuthology.orchestra.run.smithi007.stdout:5713 punch from 0x5d5ed17 to 0x5d69212, (0xa4fb bytes) 2024-09-12T09:32:00.345 INFO:teuthology.orchestra.run.smithi007.stdout:5714 write 0x260a71c thru 0x2616f71 (0xc856 bytes) 2024-09-12T09:32:00.350 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:00.350 INFO:teuthology.orchestra.run.smithi007.stdout:5716 read 0xcd807f5 thru 0xcd8499c (0x41a8 bytes) 2024-09-12T09:32:00.350 INFO:teuthology.orchestra.run.smithi007.stdout:5717 read 0x3559173 thru 0x35623bf (0x924d bytes) 2024-09-12T09:32:00.351 INFO:teuthology.orchestra.run.smithi007.stdout:5718 write 0x12083f0 thru 0x12139a9 (0xb5ba bytes) 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:5721 read 0xce50809 thru 0xce5cb84 (0xc37c bytes) 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:5723 punch from 0x55796dc to 0x557e0d6, (0x49fa bytes) 2024-09-12T09:32:00.353 INFO:teuthology.orchestra.run.smithi007.stdout:5725 write 0x7d09341 thru 0x7d16738 (0xd3f8 bytes) 2024-09-12T09:32:00.358 INFO:teuthology.orchestra.run.smithi007.stdout:5726 write 0x6f799c8 thru 0x6f83a96 (0xa0cf bytes) 2024-09-12T09:32:00.378 INFO:teuthology.orchestra.run.smithi007.stdout:5727 trunc from 0xdb3bc71 to 0x4033551 2024-09-12T09:32:00.424 INFO:teuthology.orchestra.run.smithi007.stdout:5728 writesame 0xd229c18 thru 0xd2360cf data_size 0x14(0xc4b8 bytes) 2024-09-12T09:32:00.424 INFO:teuthology.orchestra.run.smithi007.stdout:5729 write 0x4cd0ef thru 0x4d70b9 (0x9fcb bytes) 2024-09-12T09:32:00.428 INFO:teuthology.orchestra.run.smithi007.stdout:5730 writesame 0x1fa714f thru 0x1fa881d data_size 0x1(0x16cf bytes) 2024-09-12T09:32:00.436 INFO:teuthology.orchestra.run.smithi007.stdout:5731 write 0x89f639f thru 0x89fc585 (0x61e7 bytes) 2024-09-12T09:32:00.439 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:00.435+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:00.443 INFO:teuthology.orchestra.run.smithi007.stdout:5732 trunc from 0xd2360d0 to 0xc52b480 2024-09-12T09:32:00.451 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:00.454 INFO:teuthology.orchestra.run.smithi007.stdout:5738 clone 52 order 23 su 524288 sc 15 2024-09-12T09:32:00.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:00.540 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:00.534+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:00.582 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.582 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:00.640 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:00.639+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:32:00.741 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:00.738+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: marking collection for removal 2024-09-12T09:32:00.746 INFO:teuthology.orchestra.run.smithi007.stdout:setting '_remove' omap key 2024-09-12T09:32:00.747 INFO:teuthology.orchestra.run.smithi007.stdout:finish_remove_pgs 3.18_head removing 3.18 2024-09-12T09:32:00.753 INFO:teuthology.orchestra.run.smithi007.stdout:Remove successful 2024-09-12T09:32:00.770 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.770 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:00.821 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone51 from 0xc52b480 (overlap 0x12d6f88) to 0xf12606 2024-09-12T09:32:00.876 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:00.874+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:00.943 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:00.938+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:00.964 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:00.964 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:00.979 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:01.044 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:01.043+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:01.065 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-12T09:32:01.145 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:01.142+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:01.158 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:01.158 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:01.172 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:01.204 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.205 INFO:teuthology.orchestra.run.smithi007.stdout:5740 read 0x80b8381 thru 0x80c837f (0xffff bytes) 2024-09-12T09:32:01.225 INFO:teuthology.orchestra.run.smithi007.stdout:5741 trunc from 0xc52b480 to 0x447a199 2024-09-12T09:32:01.246 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:01.243+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:01.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.18.0 2024-09-12T09:32:01.348 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:01.347+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:01.349 INFO:teuthology.orchestra.run.smithi007.stdout:5742 trunc from 0x447a199 to 0x9a2b902 2024-09-12T09:32:01.357 INFO:teuthology.orchestra.run.smithi007.stdout:5744 write 0x3862b43 thru 0x3870d81 (0xe23f bytes) 2024-09-12T09:32:01.359 INFO:teuthology.orchestra.run.smithi007.stdout:5745 write 0x77a1298 thru 0x77a7ec1 (0x6c2a bytes) 2024-09-12T09:32:01.362 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.362 INFO:teuthology.orchestra.run.smithi007.stdout:5747 read 0x4c3b68e thru 0x4c495d1 (0xdf44 bytes) 2024-09-12T09:32:01.363 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.363 INFO:teuthology.orchestra.run.smithi007.stdout:5754 write 0x1e27a54 thru 0x1e3410b (0xc6b8 bytes) 2024-09-12T09:32:01.368 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:01.368 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:01.377 INFO:teuthology.orchestra.run.smithi007.stdout:5755 writesame 0xacec447 thru 0xacf896c data_size 0x67(0xc526 bytes) 2024-09-12T09:32:01.377 INFO:teuthology.orchestra.run.smithi007.stdout:5756 read 0xddc227 thru 0xde307f (0x6e59 bytes) 2024-09-12T09:32:01.378 INFO:teuthology.orchestra.run.smithi007.stdout:5757 write 0x74ca6ab thru 0x74da2c0 (0xfc16 bytes) 2024-09-12T09:32:01.380 INFO:teuthology.orchestra.run.smithi007.stdout:5758 read 0x68954c9 thru 0x689f38d (0x9ec5 bytes) 2024-09-12T09:32:01.381 INFO:teuthology.orchestra.run.smithi007.stdout:5759 punch from 0x8f878f to 0x8feb82, (0x63f3 bytes) 2024-09-12T09:32:01.381 INFO:teuthology.orchestra.run.smithi007.stdout:5761 trunc from 0xacf896d to 0xaa1fb1b 2024-09-12T09:32:01.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:5763 punch from 0x2623390 to 0x2626d7d, (0x39ed bytes) 2024-09-12T09:32:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:5768 punch from 0xbfffc6 to 0xc055b6, (0x55f0 bytes) 2024-09-12T09:32:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:5770 writesame 0x39a6c66 thru 0x39b5715 data_size 0x28(0xeab0 bytes) 2024-09-12T09:32:01.391 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.391 INFO:teuthology.orchestra.run.smithi007.stdout:5773 read 0x4a44a77 thru 0x4a4a5d8 (0x5b62 bytes) 2024-09-12T09:32:01.391 INFO:teuthology.orchestra.run.smithi007.stdout:5774 writesame 0xddde6d thru 0xdeb459 data_size 0xf(0xd5ed bytes) 2024-09-12T09:32:01.395 INFO:teuthology.orchestra.run.smithi007.stdout:5775 writesame 0x8754fb1 thru 0x875ff9a data_size 0x59(0xafea bytes) 2024-09-12T09:32:01.399 INFO:teuthology.orchestra.run.smithi007.stdout:5776 writesame 0x2fbb36e thru 0x2fbc1f6 data_size 0x1(0xe89 bytes) 2024-09-12T09:32:01.402 INFO:teuthology.orchestra.run.smithi007.stdout:5777 write 0x85d1ca thru 0x862f99 (0x5dd0 bytes) 2024-09-12T09:32:01.403 INFO:teuthology.orchestra.run.smithi007.stdout:5780 read 0x44a6fb5 thru 0x44aa497 (0x34e3 bytes) 2024-09-12T09:32:01.418 INFO:teuthology.orchestra.run.smithi007.stdout:5781 trunc from 0xaa1fb1b to 0x21d94bf 2024-09-12T09:32:01.437 INFO:teuthology.orchestra.run.smithi007.stdout:5783 flatten 2024-09-12T09:32:01.447 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:01.442+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:01.463 INFO:teuthology.orchestra.run.smithi007.stdout:5784 writesame 0x79fa6fe thru 0x7a05207 data_size 0x1(0xab0a bytes) 2024-09-12T09:32:01.463 INFO:teuthology.orchestra.run.smithi007.stdout:5785 punch from 0x672328e to 0x6731acd, (0xe83f bytes) 2024-09-12T09:32:01.463 INFO:teuthology.orchestra.run.smithi007.stdout:5786 punch from 0x2320906 to 0x232787b, (0x6f75 bytes) 2024-09-12T09:32:01.464 INFO:teuthology.orchestra.run.smithi007.stdout:5787 write 0x4286978 thru 0x429425c (0xd8e5 bytes) 2024-09-12T09:32:01.466 INFO:teuthology.orchestra.run.smithi007.stdout:5788 punch from 0x1f17295 to 0x1f1b6aa, (0x4415 bytes) 2024-09-12T09:32:01.466 INFO:teuthology.orchestra.run.smithi007.stdout:5790 punch from 0x651b8f8 to 0x6524d65, (0x946d bytes) 2024-09-12T09:32:01.466 INFO:teuthology.orchestra.run.smithi007.stdout:5791 punch from 0x4cf3242 to 0x4cf690f, (0x36cd bytes) 2024-09-12T09:32:01.466 INFO:teuthology.orchestra.run.smithi007.stdout:5792 read 0x3db458 thru 0x3e31ed (0x7d96 bytes) 2024-09-12T09:32:01.472 INFO:teuthology.orchestra.run.smithi007.stdout:5793 trunc from 0x7a05208 to 0x4aaf159 2024-09-12T09:32:01.486 INFO:teuthology.orchestra.run.smithi007.stdout:5794 read 0x3bd4ae thru 0x3bead3 (0x1626 bytes) 2024-09-12T09:32:01.486 INFO:teuthology.orchestra.run.smithi007.stdout:5796 punch from 0x10f4028 to 0x10f899e, (0x4976 bytes) 2024-09-12T09:32:01.496 INFO:teuthology.orchestra.run.smithi007.stdout:5797 trunc from 0x4aaf159 to 0x98313c1 2024-09-12T09:32:01.504 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:01.504 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:01.505 INFO:teuthology.orchestra.run.smithi007.stdout:5798 read 0x3e09367 thru 0x3e188b0 (0xf54a bytes) 2024-09-12T09:32:01.506 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.512 INFO:teuthology.orchestra.run.smithi007.stdout:5801 trunc from 0x98313c1 to 0xc84d668 2024-09-12T09:32:01.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:01.523 INFO:teuthology.orchestra.run.smithi007.stdout:5803 read 0x5174d55 thru 0x517e454 (0x9700 bytes) 2024-09-12T09:32:01.524 INFO:teuthology.orchestra.run.smithi007.stdout:5804 read 0x56c518f thru 0x56ce4c1 (0x9333 bytes) 2024-09-12T09:32:01.539 INFO:teuthology.orchestra.run.smithi007.stdout:5805 writesame 0xd334ff2 thru 0xd33b1f4 data_size 0x1(0x6203 bytes) 2024-09-12T09:32:01.539 INFO:teuthology.orchestra.run.smithi007.stdout:5806 write 0x4b3b688 thru 0x4b4a6b6 (0xf02f bytes) 2024-09-12T09:32:01.543 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.544 INFO:teuthology.orchestra.run.smithi007.stdout:5810 write 0x5732594 thru 0x5738cf4 (0x6761 bytes) 2024-09-12T09:32:01.545 INFO:teuthology.orchestra.run.smithi007.stdout:5811 write 0x55536af thru 0x55616e1 (0xe033 bytes) 2024-09-12T09:32:01.549 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:01.547+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:01.549 INFO:teuthology.orchestra.run.smithi007.stdout:5812 read 0x75207d thru 0x759133 (0x70b7 bytes) 2024-09-12T09:32:01.549 INFO:teuthology.orchestra.run.smithi007.stdout:5813 punch from 0x6c3914c to 0x6c4517b, (0xc02f bytes) 2024-09-12T09:32:01.549 INFO:teuthology.orchestra.run.smithi007.stdout:5814 read 0x3ebaa65 thru 0x3ebe180 (0x371c bytes) 2024-09-12T09:32:01.552 INFO:teuthology.orchestra.run.smithi007.stdout:5815 trunc from 0xd33b1f5 to 0xbf8b9b6 2024-09-12T09:32:01.564 INFO:teuthology.orchestra.run.smithi007.stdout:5816 write 0x18fe1fb thru 0x19024b1 (0x42b7 bytes) 2024-09-12T09:32:01.581 INFO:teuthology.orchestra.run.smithi007.stdout:5817 trunc from 0xbf8b9b6 to 0x3c576bf 2024-09-12T09:32:01.605 INFO:teuthology.orchestra.run.smithi007.stdout:5819 trunc from 0x3c576bf to 0x1a6cd6c 2024-09-12T09:32:01.628 INFO:teuthology.orchestra.run.smithi007.stdout:5821 read 0x12abdff thru 0x12b1587 (0x5789 bytes) 2024-09-12T09:32:01.629 INFO:teuthology.orchestra.run.smithi007.stdout:5822 read 0x99c18 thru 0xa53a2 (0xb78b bytes) 2024-09-12T09:32:01.632 INFO:teuthology.orchestra.run.smithi007.stdout:5823 trunc from 0x1a6cd6c to 0xd5b4a1 2024-09-12T09:32:01.650 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:01.646+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:01.663 INFO:teuthology.orchestra.run.smithi007.stdout:5825 write 0x4e9c74 thru 0x4f7a09 (0xdd96 bytes) 2024-09-12T09:32:01.666 INFO:teuthology.orchestra.run.smithi007.stdout:5826 read 0x324fc3 thru 0x32668b (0x16c9 bytes) 2024-09-12T09:32:01.667 INFO:teuthology.orchestra.run.smithi007.stdout:5827 read 0x53ba98 thru 0x549eda (0xe443 bytes) 2024-09-12T09:32:01.669 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.670 INFO:teuthology.orchestra.run.smithi007.stdout:5829 trunc from 0xd5b4a1 to 0xf94645 2024-09-12T09:32:01.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:01.705 INFO:teuthology.orchestra.run.smithi007.stdout:5830 write 0x5528148 thru 0x55323ea (0xa2a3 bytes) 2024-09-12T09:32:01.709 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.709 INFO:teuthology.orchestra.run.smithi007.stdout:5832 write 0xcb4f0f thru 0xcb97bd (0x48af bytes) 2024-09-12T09:32:01.713 INFO:teuthology.orchestra.run.smithi007.stdout:5833 write 0x2ba1bae thru 0x2bb1926 (0xfd79 bytes) 2024-09-12T09:32:01.730 INFO:teuthology.orchestra.run.smithi007.stdout:5835 writesame 0x569fa6a thru 0x56a36b9 data_size 0x28(0x3c50 bytes) 2024-09-12T09:32:01.732 INFO:teuthology.orchestra.run.smithi007.stdout:5836 write 0x3813c0a thru 0x381fbfa (0xbff1 bytes) 2024-09-12T09:32:01.750 INFO:teuthology.orchestra.run.smithi007.stdout:5837 write 0x5d2cdcb thru 0x5d2f9c2 (0x2bf8 bytes) 2024-09-12T09:32:01.751 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:01.747+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:01.754 INFO:teuthology.orchestra.run.smithi007.stdout:5839 write 0x2624e01 thru 0x2625ef6 (0x10f6 bytes) 2024-09-12T09:32:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:5840 punch from 0x1d06098 to 0x1d1088a, (0xa7f2 bytes) 2024-09-12T09:32:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:5842 punch from 0x344cf2b to 0x3456806, (0x98db bytes) 2024-09-12T09:32:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:5843 read 0x3c38aae thru 0x3c40075 (0x75c8 bytes) 2024-09-12T09:32:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:5844 write 0x40c118a thru 0x40cea63 (0xd8da bytes) 2024-09-12T09:32:01.760 INFO:teuthology.orchestra.run.smithi007.stdout:5845 punch from 0x29510dd to 0x295fa5e, (0xe981 bytes) 2024-09-12T09:32:01.783 INFO:teuthology.orchestra.run.smithi007.stdout:5846 write 0xcd5afb8 thru 0xcd60962 (0x59ab bytes) 2024-09-12T09:32:01.791 INFO:teuthology.orchestra.run.smithi007.stdout:5847 write 0x90ba2b5 thru 0x90c1793 (0x74df bytes) 2024-09-12T09:32:01.798 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.798 INFO:teuthology.orchestra.run.smithi007.stdout:5849 write 0x771443 thru 0x777dfe (0x69bc bytes) 2024-09-12T09:32:01.801 INFO:teuthology.orchestra.run.smithi007.stdout:5850 punch from 0x6c0977 to 0x6c1cb4, (0x133d bytes) 2024-09-12T09:32:01.801 INFO:teuthology.orchestra.run.smithi007.stdout:5851 punch from 0x473b7f to 0x479291, (0x5712 bytes) 2024-09-12T09:32:01.801 INFO:teuthology.orchestra.run.smithi007.stdout:5852 write 0xc3d0ae1 thru 0xc3dc6dc (0xbbfc bytes) 2024-09-12T09:32:01.809 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:01.809 INFO:teuthology.orchestra.run.smithi007.stdout:5854 writesame 0x1918c46 thru 0x1924d3a data_size 0x1(0xc0f5 bytes) 2024-09-12T09:32:01.842 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:01.842 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:01.852 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:01.850+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:01.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:01.869 INFO:teuthology.orchestra.run.smithi007.stdout:5856 write 0x905253f thru 0x905c95c (0xa41e bytes) 2024-09-12T09:32:01.877 INFO:teuthology.orchestra.run.smithi007.stdout:5857 trunc from 0xcd60963 to 0xd320ede 2024-09-12T09:32:01.891 INFO:teuthology.orchestra.run.smithi007.stdout:5860 read 0x229babd thru 0x229bbcf (0x113 bytes) 2024-09-12T09:32:01.892 INFO:teuthology.orchestra.run.smithi007.stdout:5861 writesame 0xa6b781e thru 0xa6bb06f data_size 0x9(0x3852 bytes) 2024-09-12T09:32:01.896 INFO:teuthology.orchestra.run.smithi007.stdout:5863 write 0x10502be thru 0x1056d00 (0x6a43 bytes) 2024-09-12T09:32:01.909 INFO:teuthology.orchestra.run.smithi007.stdout:5865 trunc from 0xd320ede to 0x85e2efe 2024-09-12T09:32:01.925 INFO:teuthology.orchestra.run.smithi007.stdout:5866 read 0x5b86761 thru 0x5b873a6 (0xc46 bytes) 2024-09-12T09:32:01.926 INFO:teuthology.orchestra.run.smithi007.stdout:5867 read 0x3220ef9 thru 0x3223458 (0x2560 bytes) 2024-09-12T09:32:01.927 INFO:teuthology.orchestra.run.smithi007.stdout:5868 read 0x2f5cc95 thru 0x2f5e99e (0x1d0a bytes) 2024-09-12T09:32:01.928 INFO:teuthology.orchestra.run.smithi007.stdout:5869 writesame 0x56d0a88 thru 0x56d7b0d data_size 0x3(0x7086 bytes) 2024-09-12T09:32:01.939 INFO:teuthology.orchestra.run.smithi007.stdout:5872 writesame 0x46523f8 thru 0x465446b data_size 0x1f(0x2074 bytes) 2024-09-12T09:32:01.943 INFO:teuthology.orchestra.run.smithi007.stdout:5873 write 0x8543336 thru 0x85452e2 (0x1fad bytes) 2024-09-12T09:32:01.952 INFO:teuthology.orchestra.run.smithi007.stdout:5874 writesame 0x2c75ab1 thru 0x2c7a394 data_size 0x1e(0x48e4 bytes) 2024-09-12T09:32:01.953 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:01.950+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:01.957 INFO:teuthology.orchestra.run.smithi007.stdout:5875 writesame 0x64f1797 thru 0x65004d6 data_size 0x20(0xed40 bytes) 2024-09-12T09:32:01.983 INFO:teuthology.orchestra.run.smithi007.stdout:5876 write 0xb64b542 thru 0xb64e46c (0x2f2b bytes) 2024-09-12T09:32:01.986 INFO:teuthology.orchestra.run.smithi007.stdout:5878 read 0x9941626 thru 0x9945f68 (0x4943 bytes) 2024-09-12T09:32:01.994 INFO:teuthology.orchestra.run.smithi007.stdout:5879 trunc from 0xb64e46d to 0x654bd70 2024-09-12T09:32:02.012 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:02.012 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:02.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:02.038 INFO:teuthology.orchestra.run.smithi007.stdout:5881 trunc from 0x654bd70 to 0xc21dcb2 2024-09-12T09:32:02.051 INFO:teuthology.orchestra.run.smithi007.stdout:5884 writesame 0x3a69d3c thru 0x3a78a8a data_size 0x1(0xed4f bytes) 2024-09-12T09:32:02.053 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:02.051+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:02.115 INFO:teuthology.orchestra.run.smithi007.stdout:5886 read 0x1cda9ec thru 0x1ce53d3 (0xa9e8 bytes) 2024-09-12T09:32:02.121 INFO:teuthology.orchestra.run.smithi007.stdout:5887 trunc from 0xc21dcb2 to 0x8fa5972 2024-09-12T09:32:02.122 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:02.122 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:02.133 INFO:teuthology.orchestra.run.smithi007.stdout:5888 read 0x26fcb13 thru 0x26fcb95 (0x83 bytes) 2024-09-12T09:32:02.141 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:02.142 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:02.145 INFO:teuthology.orchestra.run.smithi007.stdout:5889 write 0xa55f401 thru 0xa56a86d (0xb46d bytes) 2024-09-12T09:32:02.153 INFO:teuthology.orchestra.run.smithi007.stdout:5890 write 0x24acaf4 thru 0x24bca23 (0xff30 bytes) 2024-09-12T09:32:02.154 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:02.150+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:02.156 INFO:teuthology.orchestra.run.smithi007.stdout:5891 writesame 0x84c084f thru 0x84cccbf data_size 0x1(0xc471 bytes) 2024-09-12T09:32:02.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:02.171 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:02.171 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:02.171 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:02.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:02.188 INFO:teuthology.orchestra.run.smithi007.stdout:5894 read 0x3d89375 thru 0x3d8e200 (0x4e8c bytes) 2024-09-12T09:32:02.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:02.191 INFO:teuthology.orchestra.run.smithi007.stdout:5895 punch from 0x4e8f33d to 0x4e953d0, (0x6093 bytes) 2024-09-12T09:32:02.191 INFO:teuthology.orchestra.run.smithi007.stdout:5897 read 0x4923ecb thru 0x492bdf1 (0x7f27 bytes) 2024-09-12T09:32:02.191 INFO:teuthology.orchestra.run.smithi007.stdout:5899 read 0x48677ef thru 0x4871173 (0x9985 bytes) 2024-09-12T09:32:02.206 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:02.207 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:02.213 INFO:teuthology.orchestra.run.smithi007.stdout:5900 writesame 0xcab7694 thru 0xcac2379 data_size 0x9(0xace6 bytes) 2024-09-12T09:32:02.216 INFO:teuthology.orchestra.run.smithi007.stdout:5907 write 0x622214 thru 0x62c126 (0x9f13 bytes) 2024-09-12T09:32:02.220 INFO:teuthology.orchestra.run.smithi007.stdout:5908 writesame 0x8e36a40 thru 0x8e3e822 data_size 0x25(0x7de3 bytes) 2024-09-12T09:32:02.228 INFO:teuthology.orchestra.run.smithi007.stdout:5909 punch from 0x62fe635 to 0x6309688, (0xb053 bytes) 2024-09-12T09:32:02.228 INFO:teuthology.orchestra.run.smithi007.stdout:5910 punch from 0xbd9ca1e to 0xbda28b3, (0x5e95 bytes) 2024-09-12T09:32:02.228 INFO:teuthology.orchestra.run.smithi007.stdout:5913 write 0x2032cfd thru 0x203cd59 (0xa05d bytes) 2024-09-12T09:32:02.230 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:02.231 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:02.252 INFO:teuthology.orchestra.run.smithi007.stdout:5915 trunc from 0xcac237a to 0xb42b47 2024-09-12T09:32:02.254 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:02.251+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:02.264 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:02.269 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:02.269 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:02.286 INFO:teuthology.orchestra.run.smithi007.stdout:5917 read 0x6ee00d thru 0x6f3d4d (0x5d41 bytes) 2024-09-12T09:32:02.287 INFO:teuthology.orchestra.run.smithi007.stdout:5918 punch from 0x9eb404 to 0x9eeb2f, (0x372b bytes) 2024-09-12T09:32:02.311 INFO:teuthology.orchestra.run.smithi007.stdout:5919 write 0x9ba3b4b thru 0x9ba7f83 (0x4439 bytes) 2024-09-12T09:32:02.317 INFO:teuthology.orchestra.run.smithi007.stdout:5920 punch from 0x596228 to 0x5a25bb, (0xc393 bytes) 2024-09-12T09:32:02.317 INFO:teuthology.orchestra.run.smithi007.stdout:5921 read 0x5f6cef3 thru 0x5f76cb9 (0x9dc7 bytes) 2024-09-12T09:32:02.318 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:02.318 INFO:teuthology.orchestra.run.smithi007.stdout:5923 punch from 0x32f7b52 to 0x3303d30, (0xc1de bytes) 2024-09-12T09:32:02.318 INFO:teuthology.orchestra.run.smithi007.stdout:5924 write 0x1c8b27f thru 0x1c8c4b9 (0x123b bytes) 2024-09-12T09:32:02.321 INFO:teuthology.orchestra.run.smithi007.stdout:5925 clone 53 order 18 su 4096 sc 3 2024-09-12T09:32:02.327 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:02.327 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:02.327 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:02.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:02.355 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:02.350+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:02.457 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:02.455+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.634560519000004, 2024-09-12T09:32:02.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:02.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:02.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.287030630999993, 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.284576720999993, 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:02.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:02.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.282466834000004, 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:02.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.786720188999993, 2024-09-12T09:32:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:02.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:02.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558945672999997, 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558885111000002, 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:02.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558820834000002, 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:02.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:02.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:02.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558753314999997, 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558648689000002, 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:02.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:02.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558585157000003, 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.573 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558547216999997, 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:02.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558518696999997, 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558469193999997, 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558424805000001, 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558362922000001, 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558315409000002, 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558287139000001, 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558259585999998, 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.558228802999999, 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:02.625 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:02.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:02.627 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:02.623+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:02.727 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:02.727+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:02.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:32:02.801 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:02.802 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:02.802 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:02.802 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:02.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:02.863 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:02.863+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:02.929 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:02.926+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:02.946 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:02.946 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:02.946 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:02.946 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:02.946 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:02.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:get_pg_num_history pg_num_history pg_num_history(e140 pg_nums {1={15=1},2={18=8},3={23=32}} deleted_pools ) 2024-09-12T09:32:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:Importing pgid 3.18 2024-09-12T09:32:03.023 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:182d8453:::rbd_data.1191f0cca7c3.000000000000001b:head# 2024-09-12T09:32:03.024 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 13=[]:{} 2024-09-12T09:32:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:188bf538:::rbd_data.11914e57135.0000000000000028:head# 2024-09-12T09:32:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{} 2024-09-12T09:32:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:188da08b:::rbd_object_map.11912e4e65e9.0000000000000006:head# 2024-09-12T09:32:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.026 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:18910b52:::rbd_data.1191e0d4a936.000000000000000b:head# 2024-09-12T09:32:03.027 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.030 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:03.026+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1891e83f:::rbd_data.11914e57135.000000000000004f:head# 2024-09-12T09:32:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{} 2024-09-12T09:32:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:18d14836:::rbd_data.1191b7249abe.000000000000016d:20# 2024-09-12T09:32:03.032 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:18d14836:::rbd_data.1191b7249abe.000000000000016d:head# 2024-09-12T09:32:03.032 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 20=[]:{20=[20]} 2024-09-12T09:32:03.033 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:193bbcf1:::rbd_id.image_client.0-clone20:head# 2024-09-12T09:32:03.033 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.033 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:194660e1:::rbd_data.1191f7cb0ecd.0000000000000001:head# 2024-09-12T09:32:03.033 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.034 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19bc2ff7:::rbd_data.11918dbccc64.0000000000000004:f# 2024-09-12T09:32:03.034 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19bc2ff7:::rbd_data.11918dbccc64.0000000000000004:head# 2024-09-12T09:32:03.034 INFO:teuthology.orchestra.run.smithi007.stdout:snapset f=[]:{f=[f]} 2024-09-12T09:32:03.034 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19c03697:::rbd_data.11914e57135.0000000000000010:b# 2024-09-12T09:32:03.036 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19c03697:::rbd_data.11914e57135.0000000000000010:head# 2024-09-12T09:32:03.036 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{b=[b]} 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19cfe708:::rbd_data.1191fc712d4f.000000000000000a:head# 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 4=[]:{} 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:19fab269:::rbd_header.11914f8d16a3:head# 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a1aabe5:::rbd_data.1191f7cb0ecd.0000000000000007:head# 2024-09-12T09:32:03.037 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.038 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a57b117:::rbd_data.11914330b43e.000000000000001c:9# 2024-09-12T09:32:03.038 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a57b117:::rbd_data.11914330b43e.000000000000001c:head# 2024-09-12T09:32:03.039 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 9=[]:{9=[9]} 2024-09-12T09:32:03.039 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a5cda46:::rbd_data.11914330b43e.0000000000000023:head# 2024-09-12T09:32:03.039 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 9=[]:{} 2024-09-12T09:32:03.039 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a633d33:::rbd_data.11917d6cdfd3.0000000000000019:1c# 2024-09-12T09:32:03.040 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1a633d33:::rbd_data.11917d6cdfd3.0000000000000019:head# 2024-09-12T09:32:03.041 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 1c=[]:{1c=[1c]} 2024-09-12T09:32:03.041 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1b8a554d:::rbd_data.1191d92ee0a3.0000000000000022:head# 2024-09-12T09:32:03.041 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.042 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c139081:::rbd_data.1191b7249abe.000000000000016c:20# 2024-09-12T09:32:03.042 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c139081:::rbd_data.1191b7249abe.000000000000016c:head# 2024-09-12T09:32:03.042 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 20=[]:{20=[20]} 2024-09-12T09:32:03.043 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c43210e:::rbd_object_map.1191410774a4:head# 2024-09-12T09:32:03.043 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.043 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c45e6b1:::rbd_data.1191d92ee0a3.0000000000000068:head# 2024-09-12T09:32:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c7beb88:::rbd_data.1191ea5030de.0000000000000073:1a# 2024-09-12T09:32:03.045 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1c7beb88:::rbd_data.1191ea5030de.0000000000000073:head# 2024-09-12T09:32:03.045 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 1a=[]:{1a=[1a]} 2024-09-12T09:32:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1cbfaa64:::rbd_data.1191b7249abe.0000000000000006:head# 2024-09-12T09:32:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1cc161c0:::rbd_data.119172280d2a.0000000000000007:head# 2024-09-12T09:32:03.046 INFO:teuthology.orchestra.run.smithi007.stdout:snapset c=[]:{} 2024-09-12T09:32:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1cf74962:::rbd_data.11914da40b12.0000000000000017:16# 2024-09-12T09:32:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1cf74962:::rbd_data.11914da40b12.0000000000000017:head# 2024-09-12T09:32:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 16=[]:{16=[16]} 2024-09-12T09:32:03.048 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d31213f:::rbd_data.11912e4e65e9.000000000000005e:head# 2024-09-12T09:32:03.048 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.048 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d4f728d:::rbd_data.11914f8d16a3.0000000000000009:30# 2024-09-12T09:32:03.049 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d4f728d:::rbd_data.11914f8d16a3.0000000000000009:head# 2024-09-12T09:32:03.049 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 30=[]:{30=[30]} 2024-09-12T09:32:03.050 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d5d3248:::rbd_data.11914e57135.0000000000000003:b# 2024-09-12T09:32:03.052 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d5d3248:::rbd_data.11914e57135.0000000000000003:head# 2024-09-12T09:32:03.053 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{b=[b]} 2024-09-12T09:32:03.053 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d6ddbf2:::rbd_data.1191ec363dbc.0000000000000003:head# 2024-09-12T09:32:03.053 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.054 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-09-12T09:32:03.055 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1d7fd0d3:::rbd_data.11914330b43e.000000000000004c:head# 2024-09-12T09:32:03.055 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 9=[]:{} 2024-09-12T09:32:03.055 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1da5b711:::rbd_data.119195859761.0000000000000002:1d# 2024-09-12T09:32:03.056 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1da5b711:::rbd_data.119195859761.0000000000000002:head# 2024-09-12T09:32:03.056 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 1d=[]:{1d=[1d]} 2024-09-12T09:32:03.057 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1db9b28f:::rbd_data.11914e57135.000000000000004c:head# 2024-09-12T09:32:03.057 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{} 2024-09-12T09:32:03.057 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1dcd4cda:::rbd_data.1191b7249abe.000000000000005f:20# 2024-09-12T09:32:03.058 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1dcd4cda:::rbd_data.1191b7249abe.000000000000005f:head# 2024-09-12T09:32:03.058 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 20=[]:{20=[20]} 2024-09-12T09:32:03.058 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1de54f48:::rbd_data.11914e57135.000000000000003a:head# 2024-09-12T09:32:03.058 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{} 2024-09-12T09:32:03.059 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1ded4b08:::rbd_data.1191c1f77e6b.0000000000000042:head# 2024-09-12T09:32:03.059 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.059 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e07f715:::rbd_data.11914330b43e.000000000000001d:9# 2024-09-12T09:32:03.060 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e07f715:::rbd_data.11914330b43e.000000000000001d:head# 2024-09-12T09:32:03.060 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 9=[]:{9=[9]} 2024-09-12T09:32:03.060 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e1cc104:::rbd_data.11914da40b12.0000000000000002:16# 2024-09-12T09:32:03.061 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e1cc104:::rbd_data.11914da40b12.0000000000000002:head# 2024-09-12T09:32:03.061 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 16=[]:{16=[16]} 2024-09-12T09:32:03.061 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e78e1b0:::rbd_data.119154ef11c8.0000000000000002:2f# 2024-09-12T09:32:03.073 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1e78e1b0:::rbd_data.119154ef11c8.0000000000000002:head# 2024-09-12T09:32:03.074 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 2f=[]:{2f=[2f]} 2024-09-12T09:32:03.075 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1ec02b2f:::rbd_data.1191915fc47e.0000000000000018:head# 2024-09-12T09:32:03.075 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 32=[]:{} 2024-09-12T09:32:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1ed43b80:::rbd_data.11914e57135.000000000000002a:head# 2024-09-12T09:32:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:snapset b=[]:{} 2024-09-12T09:32:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1f37ac9b:::rbd_data.11913fee8ce1.0000000000000010:head# 2024-09-12T09:32:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 8=[]:{} 2024-09-12T09:32:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1f46b0e2:::rbd_data.1191f0cca7c3.0000000000000025:13# 2024-09-12T09:32:03.079 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1f46b0e2:::rbd_data.1191f0cca7c3.0000000000000025:head# 2024-09-12T09:32:03.079 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 13=[]:{13=[13]} 2024-09-12T09:32:03.079 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1f942b46:::rbd_data.11914f8d16a3.0000000000000011:30# 2024-09-12T09:32:03.082 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1f942b46:::rbd_data.11914f8d16a3.0000000000000011:head# 2024-09-12T09:32:03.082 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 30=[]:{30=[30]} 2024-09-12T09:32:03.082 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1feb8588:::rbd_id.image_client.0-clone41:head# 2024-09-12T09:32:03.083 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.083 INFO:teuthology.orchestra.run.smithi007.stdout:Write #3:1ff21f0c:::rbd_object_map.1191e31dd75:head# 2024-09-12T09:32:03.083 INFO:teuthology.orchestra.run.smithi007.stdout:snapset 0=[]:{} 2024-09-12T09:32:03.083 INFO:teuthology.orchestra.run.smithi007.stdout:write_pg epoch 140 info 3.18( v 133'378 (0'0,133'378] local-lis/les=0/0 n=61 ec=23/23 lis/c=136/128 les/c/f=137/129/0 sis=140) 2024-09-12T09:32:03.085 INFO:teuthology.orchestra.run.smithi007.stdout:Import successful 2024-09-12T09:32:03.112 INFO:teuthology.orchestra.run.smithi007.stdout:5926 write 0xaa07c79 thru 0xaa08496 (0x81e bytes) 2024-09-12T09:32:03.130 INFO:teuthology.orchestra.run.smithi007.stdout:5927 write 0xb3f5ebb thru 0xb3fb2d3 (0x5419 bytes) 2024-09-12T09:32:03.130 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:03.127+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:03.141 INFO:teuthology.orchestra.run.smithi007.stdout:5928 punch from 0x6b2598b to 0x6b2de55, (0x84ca bytes) 2024-09-12T09:32:03.141 INFO:teuthology.orchestra.run.smithi007.stdout:5929 write 0x3ccad63 thru 0x3cd470b (0x99a9 bytes) 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.152 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.715769813999998, 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:03.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.715744583000003, 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:03.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.715706662999999, 2024-09-12T09:32:03.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:03.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:03.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.167 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.715676662, 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:03.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.707642241000002, 2024-09-12T09:32:03.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:03.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:03.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.707321368000002, 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:03.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:03.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.706846063, 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:03.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.706817315000002, 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.187 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:03.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.706266849000002, 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:03.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:03.196 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.706224026999998, 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:03.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:03.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.706178135999998, 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:03.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.705666577999999, 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.705618799, 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:03.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.705563153, 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.215 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:03.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.705531131999997, 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:03.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:03.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.704966982999998, 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:03.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:03.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:03.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.704573082000003, 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:03.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.704181384000002, 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:03.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.336315266, 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:03.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:03.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.47783847, 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:03.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:03.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:03.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:03.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:03.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:03.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:03.251 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:03.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:03.253 INFO:teuthology.orchestra.run.smithi007.stdout:5931 punch from 0x1103df0 to 0x110f62d, (0xb83d bytes) 2024-09-12T09:32:03.253 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.253 INFO:teuthology.orchestra.run.smithi007.stdout:5934 write 0xb7b15a6 thru 0xb7bd685 (0xc0e0 bytes) 2024-09-12T09:32:03.254 INFO:teuthology.orchestra.run.smithi007.stdout:5935 read 0x2ea34be thru 0x2ea7029 (0x3b6c bytes) 2024-09-12T09:32:03.254 INFO:teuthology.orchestra.run.smithi007.stdout:5936 punch from 0x7d4435a to 0x7d47d54, (0x39fa bytes) 2024-09-12T09:32:03.254 INFO:teuthology.orchestra.run.smithi007.stdout:5937 punch from 0x6316d57 to 0x6317213, (0x4bc bytes) 2024-09-12T09:32:03.254 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.254 INFO:teuthology.orchestra.run.smithi007.stdout:5939 trunc from 0xb7bd686 to 0x1458d19 2024-09-12T09:32:03.254 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:03.251+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:03.353 INFO:teuthology.orchestra.run.smithi007.stdout:5940 writesame 0x6478698 thru 0x647b9fe data_size 0x1(0x3367 bytes) 2024-09-12T09:32:03.353 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:03.351+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:03.363 INFO:teuthology.orchestra.run.smithi007.stdout:5942 trunc from 0x647b9ff to 0xa67983b 2024-09-12T09:32:03.375 INFO:teuthology.orchestra.run.smithi007.stdout:5943 read 0x531773d thru 0x531fbda (0x849e bytes) 2024-09-12T09:32:03.375 INFO:teuthology.orchestra.run.smithi007.stdout:5944 read 0x7f904ca thru 0x7f92f54 (0x2a8b bytes) 2024-09-12T09:32:03.375 INFO:teuthology.orchestra.run.smithi007.stdout:5947 punch from 0x75ea63d to 0x75eacb7, (0x67a bytes) 2024-09-12T09:32:03.375 INFO:teuthology.orchestra.run.smithi007.stdout:5948 read 0x68ec40a thru 0x68f7c69 (0xb860 bytes) 2024-09-12T09:32:03.375 INFO:teuthology.orchestra.run.smithi007.stdout:5949 read 0x1e8f486 thru 0x1e95b36 (0x66b1 bytes) 2024-09-12T09:32:03.376 INFO:teuthology.orchestra.run.smithi007.stdout:5950 writesame 0x9edaa9b thru 0x9ee9c0f data_size 0x1(0xf175 bytes) 2024-09-12T09:32:03.392 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:03.392 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:03.392 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:03.392 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:03.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:03.426 INFO:teuthology.orchestra.run.smithi007.stdout:5951 trunc from 0xa67983b to 0xbd4af9a 2024-09-12T09:32:03.436 INFO:teuthology.orchestra.run.smithi007.stdout:5953 punch from 0x136e759 to 0x13717a3, (0x304a bytes) 2024-09-12T09:32:03.436 INFO:teuthology.orchestra.run.smithi007.stdout:5954 read 0x71070ec thru 0x7116df7 (0xfd0c bytes) 2024-09-12T09:32:03.436 INFO:teuthology.orchestra.run.smithi007.stdout:5955 write 0x96d0bc4 thru 0x96db03b (0xa478 bytes) 2024-09-12T09:32:03.455 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:03.451+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:03.462 INFO:teuthology.orchestra.run.smithi007.stdout:5957 write 0xd94c153 thru 0xd94f068 (0x2f16 bytes) 2024-09-12T09:32:03.483 INFO:teuthology.orchestra.run.smithi007.stdout:5958 trunc from 0xd94f069 to 0x9c23434 2024-09-12T09:32:03.508 INFO:teuthology.orchestra.run.smithi007.stdout:5959 read 0x7e9034 thru 0x7ea28c (0x1259 bytes) 2024-09-12T09:32:03.510 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.532 INFO:teuthology.orchestra.run.smithi007.stdout:5963 writesame 0xccf9571 thru 0xccf9862 data_size 0x2(0x2f2 bytes) 2024-09-12T09:32:03.533 INFO:teuthology.orchestra.run.smithi007.stdout:5964 punch from 0xa5ac133 to 0xa5b0a82, (0x494f bytes) 2024-09-12T09:32:03.533 INFO:teuthology.orchestra.run.smithi007.stdout:5965 write 0x1715a17 thru 0x171bf11 (0x64fb bytes) 2024-09-12T09:32:03.539 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:03.539 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:03.540 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:03.540 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:03.540 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:03.554 INFO:teuthology.orchestra.run.smithi007.stdout:5966 trunc from 0xccf9863 to 0xa0d0870 2024-09-12T09:32:03.555 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:03.555 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:03.550+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:03.568 INFO:teuthology.orchestra.run.smithi007.stdout:5967 writesame 0x8be5f5f thru 0x8bed166 data_size 0x59(0x7208 bytes) 2024-09-12T09:32:03.578 INFO:teuthology.orchestra.run.smithi007.stdout:5969 punch from 0x13b01f3 to 0x13bd08c, (0xce99 bytes) 2024-09-12T09:32:03.578 INFO:teuthology.orchestra.run.smithi007.stdout:5970 read 0x6d61aaf thru 0x6d69306 (0x7858 bytes) 2024-09-12T09:32:03.602 INFO:teuthology.orchestra.run.smithi007.stdout:5971 writesame 0xe3c5c83 thru 0xe3c87c4 data_size 0x31(0x2b42 bytes) 2024-09-12T09:32:03.602 INFO:teuthology.orchestra.run.smithi007.stdout:5973 write 0x70f68de thru 0x7104dbc (0xe4df bytes) 2024-09-12T09:32:03.614 INFO:teuthology.orchestra.run.smithi007.stdout:5974 write 0x9adeeab thru 0x9ae3332 (0x4488 bytes) 2024-09-12T09:32:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:5975 read 0x5a4b651 thru 0x5a52ecd (0x787d bytes) 2024-09-12T09:32:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:5976 read 0x544588e thru 0x54458cd (0x40 bytes) 2024-09-12T09:32:03.624 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /var/log/ceph/exp.3.18.0 2024-09-12T09:32:03.657 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:03.654+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:03.657 INFO:teuthology.orchestra.run.smithi007.stdout:5979 trunc from 0xe3c87c5 to 0x7a9513 2024-09-12T09:32:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 119.445572721, 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.986076492, 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.612659541, 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:32:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.612612294, 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.612319218, 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.611757286, 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.611705923, 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.611133043, 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:03.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:03.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.610980744, 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.61062365399999, 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:03.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:03.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.609998684, 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:03.797 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:03.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:03.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.60994734800001, 2024-09-12T09:32:03.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:03.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:03.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:03.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.60915255800001, 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:03.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:03.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.609077668, 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:03.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:03.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:03.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.609025348, 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:03.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.60897675, 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:03.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:03.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.608722285, 2024-09-12T09:32:03.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:03.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.60526799500001, 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 100.547699448, 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:03.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:03.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:03.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.076428211000007, 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:03.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:03.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:03.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:03.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:03.858 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:03.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:03.861 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:03.854+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5980 trunc from 0x7a9513 to 0x74c5688 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5981 trunc from 0x74c5688 to 0x32e4d2f 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5983 read 0x278693b thru 0x278b69d (0x4d63 bytes) 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5985 write 0x4d8904c thru 0x4d9004d (0x7002 bytes) 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5986 writesame 0x9ed7168 thru 0x9ee023e data_size 0x1(0x90d7 bytes) 2024-09-12T09:32:03.862 INFO:teuthology.orchestra.run.smithi007.stdout:5987 write 0x73a8667 thru 0x73b11ec (0x8b86 bytes) 2024-09-12T09:32:03.863 INFO:teuthology.orchestra.run.smithi007.stdout:5988 writesame 0x4e5a142 thru 0x4e67171 data_size 0x8(0xd030 bytes) 2024-09-12T09:32:03.863 INFO:teuthology.orchestra.run.smithi007.stdout:5989 trunc from 0x9ee023f to 0x5081bfd 2024-09-12T09:32:03.885 INFO:teuthology.orchestra.run.smithi007.stdout:5990 write 0x5788e8c thru 0x57942f8 (0xb46d bytes) 2024-09-12T09:32:03.900 INFO:teuthology.orchestra.run.smithi007.stdout:5991 punch from 0x3e2f620 to 0x3e38772, (0x9152 bytes) 2024-09-12T09:32:03.900 INFO:teuthology.orchestra.run.smithi007.stdout:5994 read 0x3bf03f0 thru 0x3bfc13d (0xbd4e bytes) 2024-09-12T09:32:03.900 INFO:teuthology.orchestra.run.smithi007.stdout:5995 punch from 0x10c855c to 0x10d62f0, (0xdd94 bytes) 2024-09-12T09:32:03.928 INFO:teuthology.orchestra.run.smithi007.stdout:5996 write 0xb31bb91 thru 0xb3205ab (0x4a1b bytes) 2024-09-12T09:32:03.937 INFO:teuthology.orchestra.run.smithi007.stdout:5997 write 0x6b38c55 thru 0x6b3dd0d (0x50b9 bytes) 2024-09-12T09:32:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:5998 read 0x1518fe6 thru 0x151c271 (0x328c bytes) 2024-09-12T09:32:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-12T09:32:03.960 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:03.958+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:03.975 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #52, image image_client.0-clone52 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent53 2024-09-12T09:32:04.037 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:04.037 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:04.037 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:04.038 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:04.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:04.096 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:04.094+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:04.163 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:04.159+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:04.213 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:04.213 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:04.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:04.264 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:04.263+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:04.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.941401955, 2024-09-12T09:32:04.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:04.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:04.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.941306693, 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:04.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:04.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.93762206900001, 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.93581697900001, 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:04.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:04.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.935270304, 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:04.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.934501861, 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:04.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:04.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.93409149300001, 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.93395231700001, 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:04.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:04.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.926173506, 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:04.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:04.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.923809492, 2024-09-12T09:32:04.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:04.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:04.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.91305759900001, 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:04.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.91286120800001, 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:04.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.91136678800001, 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:04.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.211123927, 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:04.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:04.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.211101509, 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.211078368, 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:04.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.211053409, 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:04.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.16770189499999, 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:04.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:04.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.446 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 100.551223853, 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.447 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:04.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:04.452 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 100.551124563, 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:04.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:04.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:04.457 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:04.458 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:04.458 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:04.459 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:04.459 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:04.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:04.460 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:04.454+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:04.460 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:04.461 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:04.461 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:04.482 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:04.505 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:04.524 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:04.524 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:04.544 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:04.544 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:04.547 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:04.548 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:04.559 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:04.554+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:04.635 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:04.635 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:04.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:04.661 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:04.659+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:04.762 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:04.758+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:04.834 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:04.835 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:04.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:04.863 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:04.859+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:04.964 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:04.962+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:05.030 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.031 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:05.065 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:05.063+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:05.166 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:05.163+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:05.213 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.213 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:05.272 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:05.270+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:05.349 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.349 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:05.404 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:05.402+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:05.468 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:05.467+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:05.527 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.527 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:05.588 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:05.587+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:05.670 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:05.667+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:05.676 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.676 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:05.776 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:05.775+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:05.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:32:05.868 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:05.868 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:05.872 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:05.871+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:05.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:05.980 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:05.979+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:06.060 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:06.060 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:06.074 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:06.071+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:06.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:06.175 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:06.171+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:06.247 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:06.247 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:06.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:06.276 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:06.274+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:06.377 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:06.374+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:06.435 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:06.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:06.496 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:06.495+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:06.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:06.578 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:06.579 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:06.579 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:06.575+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:06.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:06.680 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:06.679+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:06.683 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:06.701 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:06.702 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:06.719 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:06.719 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:06.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:06.758 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:06.759 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:06.759 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:32:06.781 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:06.778+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:06.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:06.783 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:06.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:06.806 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:06.828 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:06.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:06.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:06.882 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:06.878+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:06.900 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:06.901 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:06.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:06.983 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:06.979+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.608887992000007, 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.608862760999997, 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.608824841000001, 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.608794840000002, 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:07.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.600760418999997, 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.600439546000004, 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:07.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:07.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:07.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.599964240999995, 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.599935493000004, 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.599385026999997, 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:07.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.599342204999999, 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:07.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.599296314, 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:07.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.598784756000001, 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:07.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.598736977000002, 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:07.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:07.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.598681330999995, 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:07.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.598649309999999, 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:07.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.598085161, 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:07.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:07.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.597691260000005, 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:07.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.597299562000003, 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:07.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.229433444000001, 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:07.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.370956648, 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:07.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:07.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:07.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:07.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:07.148 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:07.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:07.150 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:07.147+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:07.251 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:07.246+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:07.284 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:07.284 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:07.284 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:07.284 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:07.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:07.384 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:07.382+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:07.456 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:07.455+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:07.462 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:07.462 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:07.463 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:07.463 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:07.463 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:07.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:07.556 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:07.555+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:07.655 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:07.651+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:07.658 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.365730013, 2024-09-12T09:32:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.90623378399999, 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.532816833, 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:32:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:32:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:32:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:32:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:32:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:32:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:32:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.532769586, 2024-09-12T09:32:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.53247651, 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.531914578, 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.531863215, 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.53129033499999, 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.531138036, 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.53078094599999, 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.530155976, 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.53010464, 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.52930985, 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.52923496, 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.52918264, 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.529134042, 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.528879577, 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.525425287, 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:07.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.46785674, 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:07.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:07.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.996585503000006, 2024-09-12T09:32:07.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:07.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:07.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:07.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.767 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:07.768 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:07.769 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:07.767+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:07.870 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:07.867+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:198192 3.5 3:a6b15804:::rbd_id.image_client.0-clone49:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e150)", 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:07.903135+0000", 2024-09-12T09:32:07.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0025943289999999998, 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026465769999999998, 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 198192 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:07.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903135+0000", 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903135+0000", 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903138+0000", 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.357e-06 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903147+0000", 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.747e-06 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903177+0000", 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.004e-05 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903198+0000", 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1183e-05 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:07.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903362+0000", 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016354499999999999 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903570+0000", 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020826500000000001 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.903870+0000", 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029972700000000001 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:07.905310+0000", 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014399199999999999 2024-09-12T09:32:07.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:32:07.915 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:07.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:07.971 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:07.966+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:08.053 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:08.053 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:08.053 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:08.053 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:08.053 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:08.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:08.109 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:08.106+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:08.173 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:08.171+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:08.232 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:08.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.81920338099999, 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:08.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:08.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:08.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.81910811899999, 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:08.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:08.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.815423495, 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:08.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.813618405, 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.251 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:08.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:08.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.81307172999999, 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:08.257 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:08.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:08.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:08.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.81230328699999, 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:08.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.811892919, 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:08.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.275 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.811753743, 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:08.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:08.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:08.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:08.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.80397493199999, 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:08.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:08.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:08.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:08.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.80161091799999, 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:08.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.790859025, 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:08.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:08.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.790662634, 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 138.789168214, 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:08.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:08.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:08.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:08.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:08.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.08892535299999, 2024-09-12T09:32:08.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:08.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:08.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:08.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.08890293499999, 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:08.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:08.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.08887979399999, 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:08.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:08.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.08885483500001, 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:08.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:08.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:08.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.045503321, 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.429025279, 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.42892598900001, 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:08.364 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:08.365 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:08.362+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:08.465 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:08.463+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:08.534 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:08.534 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:08.535 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:08.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:08.596 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:08.594+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:08.667 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:08.667 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:08.667 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:08.667 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:08.667 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:08.667 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:08.663+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:08.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:08.683 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:08.683 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:08.683 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:32:08.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:32:08.772 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:08.770+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:08.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:32:08.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108941993999998, 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108891181000001, 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108854517000001, 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108795184000002, 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108765175999999, 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108746341, 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108719622999999, 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108698857, 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108679142, 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108648945999999, 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108618377999999, 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108599575, 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108550481000002, 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108515986, 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:08.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108483207999999, 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108448076999998, 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:08.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:08.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:08.947 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108405753, 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:08.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108370246, 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:08.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:08.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108335975999999, 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:08.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.108282638999999, 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:08.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:08.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:08.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:08.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:08.966 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:08.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:08.968 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:08.962+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:09.007 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 151, 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:32:08.769926+0000", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:31:57.668482+0000", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:51.831699+0000", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:32:09.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:32:09.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:09.036 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:32:09.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:09.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:09.041 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 7.690000057220459, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 7.690000057220459, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-12T09:32:09.042 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:09.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:32:09.044 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:09.045 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:09.046 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:09.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.049 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:09.050 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:09.051 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "151", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 55, 2024-09-12T09:32:09.052 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 151, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:09.053 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:09.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:09.055 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.9900000095367432, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.9900000095367432, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.98291778564453125, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:09.056 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 140, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 141, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:32:09.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 141, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.063 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 138, 2024-09-12T09:32:09.069 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.074 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:32:09.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:32:09.080 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.081 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:32:09.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:32:09.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.087 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:32:09.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:32:09.092 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.093 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:32:09.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:32:09.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:09.100 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 141, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:09.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 141 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:32:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.2", 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:32:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1a", 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1", 2024-09-12T09:32:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.3", 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.10", 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.17", 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:32:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 51, 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 5 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 51, 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:32:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:09.122 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:09.123 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:09.125 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:32:08.769926+0000', 'last_up_change': '2024-09-12T09:31:57.668482+0000', 'last_in_change': '2024-09-12T09:31:51.831699+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9829177856445312, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 141, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 141, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2', 'osds': [6, 7]}, {'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1', 'osds': [4, 3]}, {'pgid': '3.3', 'osds': [4, 6]}, {'pgid': '3.b', 'osds': [3, 4]}, {'pgid': '3.10', 'osds': [6, 5]}, {'pgid': '3.17', 'osds': [5, 3]}, {'pgid': '3.18', 'osds': [3, 1]}, {'pgid': '3.1b', 'osds': [4, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:32:09.125 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.2 2024-09-12T09:32:09.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.2 2024-09-12T09:32:09.126 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:09.126 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:09.127 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:09.127 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:09.127 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:09.127 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:09.128 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:09.123+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:09.137 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:09.138 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.138 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.142 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:09.142 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:09.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:09.224 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:09.223+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:09.275 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.275 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:09.325 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:09.323+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:09.414 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.414 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.425 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:09.423+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:09.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:09.526 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:09.526+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:09.549 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.549 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:09.627 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:09.626+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:09.695 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.695 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:09.756 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:09.755+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:clear 3.2 pg_upmap mapping 2024-09-12T09:32:09.778 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:32:09.829 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:09.826+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:09.843 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:09.843 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:09.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:09.944 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:09.943+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:10.024 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.024 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.030 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:10.027+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:10.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:10.131 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:10.127+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:10.212 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.212 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:10.232 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:10.230+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:10.333 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:10.331+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:10.395 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.396 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:10.452 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:10.451+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:10.534 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:10.531+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:10.577 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.577 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:10.640 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:10.639+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:10.736 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:10.735+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:10.837 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:10.834+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:10.938 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:10.938+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:10.949 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:10.949 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:10.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:11.039 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:11.035+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:11.094 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:11.094 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:11.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:11.140 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:11.139+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:11.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:11.225 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:11.225 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:11.239 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:11.241 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:11.238+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:11.281 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:11.299 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:11.316 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:11.333 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:11.341 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:11.339+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:11.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:11.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.07700493999999, 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:11.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:11.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 122.617508711, 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24409176, 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:32:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:32:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:32:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24404451300001, 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.243751437, 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.243189505, 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24313814200001, 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.242565262, 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24241296300001, 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.242055873, 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24143090299999, 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.241379567, 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.240584777, 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.240509887, 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.24045756700001, 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.240408969, 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.240154504, 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.236700214, 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 108.17913166699999, 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.707860429999997, 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:11.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.467 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.467 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:11.468 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:11.469 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:11.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:11.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:11.470 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:11.471 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:11.471 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:11.467+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:11.570 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:11.567+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:11.597 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:11.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:11.670 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:11.667+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:11.732 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:11.732 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:11.732 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:11.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:11.771 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:11.771+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:11.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.45511995000001, 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:11.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.45502468800001, 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:11.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.45134006399999, 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.44953497399999, 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.44898829900001, 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.44821985600001, 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.44780948799999, 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.44767031200001, 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.43989150100001, 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.43752748700001, 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.42677559399999, 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.42657920299999, 2024-09-12T09:32:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.42508478299999, 2024-09-12T09:32:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.724841922, 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.724819504, 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.724796363, 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.72477140399999, 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 108.68141989, 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 108.064941848, 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:11.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 108.064842558, 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:11.963 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:11.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:11.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:11.968 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:11.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:11.970 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:11.967+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:12.071 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:12.065+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:12.096 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:12.096 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:12.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:12.196 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:12.193+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:12.272 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:12.269+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:12.303 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:12.303 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:12.304 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:12.304 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:12.322 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:12.408 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:12.405+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:12.474 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:12.469+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:12.490 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734220368999999, 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734169556000001, 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734132892000002, 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734073558999999, 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734043550999999, 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.734024716, 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733997998, 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733977232000001, 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733957517, 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733927320999999, 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733896753, 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.73387795, 2024-09-12T09:32:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733828855999999, 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733794361000001, 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733761583, 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733726451999999, 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:12.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:12.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:12.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:12.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733684128, 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:12.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:12.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733648621, 2024-09-12T09:32:12.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.582 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:12.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733614351, 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:12.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.733561013999999, 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:12.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:12.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:12.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:12.597 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:12.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:12.599 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:12.593+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:32:12.698 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:12.693+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:12.786 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:12.786 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:12.786 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:12.786 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:12.800 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:12.799+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:12.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:12.901 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:12.899+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:12.967 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:12.968 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:12.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:13.001 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:12.999+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:13.102 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:13.097+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:13.106 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:13.106 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:13.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:13.106 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:13.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.890957206000003, 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:13.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:13.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:13.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:13.112 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.888833587999997, 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:13.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.888813761000002, 2024-09-12T09:32:13.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:13.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:13.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.886718727999998, 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:13.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884684120999999, 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.130 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:13.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:13.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:13.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884645403999997, 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:13.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:13.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884608419999999, 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:13.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884594036000003, 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:13.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:13.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884573045000003, 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:13.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:13.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884561574000003, 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:13.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:13.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.884536089000001, 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:13.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:13.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.882348809, 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:13.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.176 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.882338441000002, 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:13.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:13.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.882312474999999, 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:13.182 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:13.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:13.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880620653000001, 2024-09-12T09:32:13.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:13.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:13.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880606096999998, 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:13.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:13.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.199 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880585727000003, 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:13.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:13.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:13.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880547098999998, 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:13.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:13.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:13.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880532576, 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:13.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 61.880522161000002, 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:13.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:13.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:13.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:13.223 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:13.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:13.225 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:13.223+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:13.326 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:13.323+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:13.363 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:13.363 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:13.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:13.464 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:13.463+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:13.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:13.528 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:13.527+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:13.549 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:13.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:13.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:13.616 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:13.616 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:13.629 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:13.625+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:13.633 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:13.650 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:13.671 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:13.672 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:13.705 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:13.730 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:13.727+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:13.735 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:13.735 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:13.744 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:13.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:13.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:13.772 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:13.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:13.830 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:13.827+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:13.894 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:13.894 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:13.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:13.932 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:13.925+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:14.032 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:14.029+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:14.080 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.081 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:14.140 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:14.139+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:14.214 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.214 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:14.234 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:14.229+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:14.335 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:14.331+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:14.391 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.392 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:14.436 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:14.433+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:14.537 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:14.535+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.575 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:14.677 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:14.675+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:14.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:32:14.739 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:14.733+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.751 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:14.807 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:14.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:14.807 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-12T09:32:14.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-12T09:32:14.855 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:14.855+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:14.918 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:14.919 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:14.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:14.970 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:14.970 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:14.970 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:14.973 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:14.974 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:14.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-12T09:32:14.984 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:14.981+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:15.041 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:15.037+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:15.063 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:15.164 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:15.161+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:15.185 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:15.186 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:15.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-12T09:32:15.238 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.238 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.242 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:15.237+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:15.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:15.337 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:15.343 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:15.343 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:15.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-12T09:32:15.344 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:15.343+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:15.416 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.416 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:15.476 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:15.473+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:15.545 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:15.541+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:15.547 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.548 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:15.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:15.554 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:15.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-12T09:32:15.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:15.648 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:15.645+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:15.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:15.708 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:15.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-12T09:32:15.728 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.728 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.742 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:15.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:15.785 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:15.781+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:15.837 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:32:15.847 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:15.841+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:15.919 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:15.920 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:15.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:15.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:15.948 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:15.947+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:15.950 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:15.968 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:15.990 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:16.011 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:16.031 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-12T09:32:16.031+0000 7fb06e49b640 0 --2- 172.21.15.7:0/3065827270 >> v2:172.21.15.7:3300/0 conn(0x7fb0680b12e0 0x7fb06813db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-12T09:32:16.049 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:16.045+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:16.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:16.054 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:16.058 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:16.058 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:16.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:16.073 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:16.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-12T09:32:16.107 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:16.108 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:16.108 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:16.108 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:16.108 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:16.121 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:16.164 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:16.161+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:16.251 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:16.247+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:16.269 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:16.269 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:16.269 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:16.269 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:16.269 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:16.270 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:16.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:16.273 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:16.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-12T09:32:16.289 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532425031999999, 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:16.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532374219000001, 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:16.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:16.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532337555000002, 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:16.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532278221999999, 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.308 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:16.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.308 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:16.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532248213999999, 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:16.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:16.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532229379, 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:16.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532202660999999, 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532181895000001, 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:16.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.53216218, 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:16.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:16.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532131983999999, 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:16.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532101416, 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:16.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:16.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532082613, 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:16.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.532033518999999, 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:16.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:16.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:16.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531999024000001, 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:16.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531966246, 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:16.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531931114999999, 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:16.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531888791, 2024-09-12T09:32:16.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:16.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531853284, 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:16.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531819014, 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.380 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:16.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:16.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:16.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 26.531765676999999, 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:16.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:16.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.389 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:16.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:16.391 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:16.389+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:16.491 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:16.485+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:16.533 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:16.534 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:16.561 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:16.575 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:16.591 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:16.591+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:16.692 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:16.689+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:16.737 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:16.738 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:16.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:16.800 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:16.797+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:16.894 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:16.889+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.717811475000005, 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:16.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:16.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.715687857000006, 2024-09-12T09:32:16.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:16.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.715668030000003, 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:16.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:16.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:16.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.713572997, 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.951 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:16.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711538390000001, 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:16.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711499673000006, 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:16.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:16.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:16.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711462689000001, 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:16.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:16.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711448305000005, 2024-09-12T09:32:16.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711427314000005, 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.979 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:16.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711415842999997, 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:16.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.711390358000003, 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:16.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:16.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:16.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.709203078000002, 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:16.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:16.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:16.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.709192709999996, 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:17.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.709166744000001, 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:17.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707474922000003, 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:17.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:17.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:17.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707460366000007, 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:17.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:17.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707439996000005, 2024-09-12T09:32:17.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:17.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:17.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707401368000006, 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.029 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707386845000002, 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:17.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:17.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.707376429999997, 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:17.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:17.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:17.045 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:17.047 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:17.041+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:17.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:17.147 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:17.147+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:17.184 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:17.184 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:17.185 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:17.185 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:17.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:17.284 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:17.283+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:17.349 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:17.347+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:17.367 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:17.367 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:17.368 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:17.368 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:17.368 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:17.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:17.476 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:17.475+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:17.551 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:17.547+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:17.558 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:17.558 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:17.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.357523687, 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:32:17.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:32:17.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.342552958, 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:32:17.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.025705954, 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:32:17.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02565959499999, 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:32:17.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:32:17.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:17.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.025552375, 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:32:17.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.025458884, 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:32:17.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.025392371, 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:32:17.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02533156200001, 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:32:17.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02527227, 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:32:17.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02521176099999, 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:32:17.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:32:17.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02513370600001, 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:17.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02504833099999, 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:32:17.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:32:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024980012, 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:32:17.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024887427, 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:32:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:32:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024831518, 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:32:17.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:32:17.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024799195, 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:32:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:32:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024765602, 2024-09-12T09:32:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:32:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:32:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.02471466, 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:32:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024616505, 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:32:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.024559411, 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:32:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:17.655 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:17.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:17.657 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:17.655+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:17.758 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:17.755+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:17.831 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:17.831 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:17.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:17.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:32:17.892 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:17.891+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:17.959 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:17.953+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:18.024 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:18.024 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:18.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:18.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:18.060 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:18.057+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:18.161 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:18.157+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:18.171 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.295594995, 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.94806510700001, 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.94561119700001, 2024-09-12T09:32:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.94350131, 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:18.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.447754665000005, 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219980149000001, 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219919587, 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.21985531, 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219787791000002, 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:18.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.216 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219683164999999, 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:18.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:18.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219619633000001, 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:18.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219581693000002, 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:18.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:18.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:18.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219553173000001, 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:18.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.235 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219503670000002, 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.236 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219459280999999, 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219397397999998, 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219349885, 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219321614999998, 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:18.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:18.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219294062000003, 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:18.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:18.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.219263279000003, 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:18.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:18.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:18.269 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:18.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:18.271 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:18.271 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:18.273 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:18.273 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:18.267+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:18.287 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:18.288 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:18.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:18.329 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:18.353 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:18.355 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:18.371 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:18.365+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:18.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:18.472 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:18.469+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:18.545 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:18.545 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:18.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:18.573 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:18.569+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:18.673 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:18.671+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:18.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:18.775 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:18.771+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:18.836 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:18.836 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:18.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:32:18.900 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:18.897+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:18.976 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:18.973+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:18.980 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:18.981 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:18.997 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:19.084 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:19.081+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:19.163 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:19.163 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:19.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:19.179 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:19.175+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:19.280 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:19.277+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:19.345 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:19.345 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:19.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:19.404 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:19.401+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:19.482 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:19.477+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:19.525 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:19.525 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:19.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:19.583 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:19.579+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:19.684 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:19.681+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:19.707 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:19.707 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:19.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:19.785 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:19.783+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:19.886 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:19.881+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:19.897 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:19.897 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:19.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:20.004 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:20.001+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:20.088 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:20.088 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:20.088 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:20.085+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:20.102 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:20.190 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:20.187+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:20.268 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:20.268 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:20.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:20.328 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:20.325+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:20.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:20.391 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:20.385+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:20.406 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:20.406 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:20.420 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:20.508 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:20.505+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:20.517 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:20.534 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:20.552 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:20.552 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:20.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:20.573 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:20.589 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:20.589 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:20.593 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:20.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:20.594 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:20.591+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:20.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:20.616 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:20.640 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:20.640 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:20.645 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:20.645 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:20.694 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:20.689+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:20.770 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.555820890000007, 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:20.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.553697271999994, 2024-09-12T09:32:20.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:20.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:20.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.553677445000005, 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:20.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.551582412000002, 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:20.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:20.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549547805000003, 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:20.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:20.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549509087999994, 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:20.798 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549472104000003, 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:20.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:20.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549457720000007, 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:20.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:20.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549436729000007, 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:20.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:20.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549425257999999, 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:20.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:20.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.549399773000005, 2024-09-12T09:32:20.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:20.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:20.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.547212493000004, 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:20.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:20.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.547202124999998, 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:20.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:20.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.547176159000003, 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:20.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:20.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545484337000005, 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:20.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.853 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545469780999994, 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:20.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545449411000007, 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:20.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545410782999994, 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:20.864 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:20.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545396260000004, 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:20.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.545385844999998, 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:20.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:20.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:20.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:20.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:20.881 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:20.882 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:20.877+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:20.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:20.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:32:20.983 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:20.977+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:21.031 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:21.032 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:21.032 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:21.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:21.084 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:21.081+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:21.183 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.183 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:21.183 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:21.183 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:21.183 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:21.185 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:21.181+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:21.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:21.285 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:21.283+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:21.319 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.11902914300001, 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:32:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:32:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.10405841399999, 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:32:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78721141, 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:32:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.787165051, 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.787057831, 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:32:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78696434, 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:32:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:32:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:32:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786897827, 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:32:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786837018, 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:32:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786777726, 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786717217, 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786639162, 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:32:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:32:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786553787, 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786485468, 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:32:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786392883, 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:32:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78633697399999, 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:32:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78630465099999, 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:32:21.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:32:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786271058, 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.786220116, 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:32:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:32:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78612196100001, 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:32:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.78606486699999, 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:32:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:32:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.413 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:21.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:21.415 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:21.411+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:21.516 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:21.513+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:21.534 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:21.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:21.535 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-12T09:32:21.535 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.038635 2024-09-12T09:32:21.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.038635240967113904 2024-09-12T09:32:21.587 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.587 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:21.587 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:21.587 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:21.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:21.648 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:21.647+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:21.717 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:21.713+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:21.722 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.723 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:21.723 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:21.723 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:21.723 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:21.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:21.818 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:21.813+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:32:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.6 primary-affinity to 0.0386352 (825312) 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.041184327, 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.693654439, 2024-09-12T09:32:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.691200529, 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.689090642, 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.193343996999999, 2024-09-12T09:32:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965569481000003, 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965508919000001, 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965444642000001, 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965377123000003, 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965272497000001, 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965208965000002, 2024-09-12T09:32:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965171024999997, 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965142505000003, 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965093002000003, 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.965048613, 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.96498673, 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.964939217000001, 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.964910947, 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.964883393999997, 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.964852610999998, 2024-09-12T09:32:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.016 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:22.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:22.018 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:22.015+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:22.119 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:22.113+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:22.192 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:22.192 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:22.192 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:22.192 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:22.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:22.251 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:22.251+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:22.320 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:22.319+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:22.335 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:22.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:22.336 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:22.336 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:22.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:22.440 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:22.439+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.082620215999995, 2024-09-12T09:32:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.082594985, 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.082557065000003, 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.082527064000004, 2024-09-12T09:32:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.074492642999999, 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.074171770000007, 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.073696464999998, 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.073667717000006, 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.556 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.073117250999999, 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.073074429000002, 2024-09-12T09:32:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.073028538000003, 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:22.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:22.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.072516980000003, 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:22.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.072469201000004, 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:22.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:22.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.072413554999997, 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:22.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.589 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.072381534000002, 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:22.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.071817385000003, 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:22.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:22.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.071423483999993, 2024-09-12T09:32:22.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:22.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:22.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:22.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.071031786000006, 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:22.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.703165667999997, 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:22.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.844688871999999, 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:22.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:22.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:22.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:22.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:22.623 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:22.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:22.625 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:22.623+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:22.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:22.726 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:22.723+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:22.756 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:22.756 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:200348 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:22.000835+0000", 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.75605487599999999, 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.75609911200000002, 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:22.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 200348 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000835+0000", 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000835+0000", 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000837+0000", 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3130000000000001e-06 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000847+0000", 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8530000000000004e-06 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:22.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000871+0000", 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4433000000000001e-05 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.000891+0000", 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9703000000000001e-05 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.001007+0000", 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011561300000000001 2024-09-12T09:32:22.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:22.001025+0000", 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8819999999999999e-05 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:32:22.761 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:22.772 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:22.820 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:22.820 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:22.827 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:22.821+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:22.837 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:22.837 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:22.854 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:22.875 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:22.875 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:22.892 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:22.892 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:22.896 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:22.896 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:22.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:22.918 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:22.928 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:22.927+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:22.937 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:22.941 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:22.941 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:23.029 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:23.027+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:23.076 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.076 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.090 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:23.136 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:23.133+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:23.210 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.211 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:23.268 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:23.265+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:23.332 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:23.331+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:23.392 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.392 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:23.452 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:23.449+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:23.529 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.529 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.534 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:23.529+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:23.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:23.635 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:23.633+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:23.709 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.709 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:23.737 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:23.735+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:23.838 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:23.833+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:23.888 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:23.888 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:23.903 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:23.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:32:23.964 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:23.963+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:24.040 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:24.037+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:24.070 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.070 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.086 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:24.141 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:24.139+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:24.242 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:24.239+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:24.250 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.250 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.264 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:24.343 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:24.339+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:24.375 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.375 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.388 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:24.444 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:24.441+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:24.510 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.510 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:24.572 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:24.571+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:24.647 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:24.641+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:24.659 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.660 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:24.748 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:24.745+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:24.847 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:24.847 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:24.849 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:24.845+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:24.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:24.912 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:32:24.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:24.949 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:24.945+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:25.000 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:25.000 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:25.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:25.068 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:25.067+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:25.101 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:25.101 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:25.122 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:25.140 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:25.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:25.148 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:25.151 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:25.145+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:25.161 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:25.161 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:25.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:25.182 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:25.203 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:25.224 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:25.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:25.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:25.276 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:25.275+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:25.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:25.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:25.299 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:25.299 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:25.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:25.352 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:25.349+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:25.453 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:25.451+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:25.482 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.606981475, 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.259451587, 2024-09-12T09:32:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.256997677, 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.25488779, 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.759141145000001, 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531366628999997, 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531306067000003, 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531241790000003, 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531174270999998, 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531069645000002, 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.531006112999997, 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530968172999998, 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530939652999997, 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530890149999998, 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530845761000002, 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530783878000001, 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530736365000003, 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530708095000001, 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530680541999999, 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.530649758999999, 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:25.577 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:25.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:25.579 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:25.573+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:25.680 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:25.679+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:25.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:25.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:25.754 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:25.754 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:25.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:25.812 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:25.811+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:25.882 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:25.877+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:25.944 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:25.945 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:25.945 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:25.945 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:25.945 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:25.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:26.008 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:26.007+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:26.083 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:26.083+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:26.106 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.668432104999994, 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.668406873999999, 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.668368954000002, 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.668338953000003, 2024-09-12T09:32:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.660304531999998, 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.659983659000005, 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.659508353999996, 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.659479606000005, 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658929139999998, 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658886318, 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658840427000001, 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658328869000002, 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658281090000003, 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658225443999996, 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.658193423, 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.657629274000001, 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.657235373000006, 2024-09-12T09:32:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.656843675000005, 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.288977557000003, 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.430500760999998, 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:26.206 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:26.207 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:26.207 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:26.201+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:26.308 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:26.305+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:26.338 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:26.338 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:26.338 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:26.338 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:26.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:26.440 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:26.437+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:26.510 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:26.505+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:26.516 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:26.516 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:26.516 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:26.516 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:26.517 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:26.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:26.611 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:26.607+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.40287318099999, 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.94337695199999, 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.569960001, 2024-09-12T09:32:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.569912754, 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56961967800001, 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.569057746, 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56900638299999, 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56843350299999, 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56828120399999, 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:26.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:26.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56792411399999, 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:26.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.567299144, 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:26.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56724780799999, 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:26.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.566453018, 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:26.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.566378128, 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:26.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:26.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56632580799999, 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:26.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56627721000001, 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:26.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:26.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.566022745, 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:26.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:26.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.56256845499999, 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:26.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:26.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:26.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:26.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.504999908, 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:26.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:26.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.03372867100001, 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:26.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:26.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:26.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:26.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:26.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:26.798 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:26.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:26.800 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:26.799+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:26.900 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:26.897+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:26.917 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:26.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:26.917 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-12T09:32:26.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-12T09:32:26.925 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:26.925 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:26.925 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:26.925 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:26.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:26.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:32:27.008 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:27.007+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:27.101 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:27.099+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:27.111 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:27.112 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:27.112 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:27.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:27.115 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:27.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-12T09:32:27.116 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:27.116 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:27.116 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:27.116 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:27.116 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:27.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:27.202 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:27.197+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:27.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.84164268399999, 2024-09-12T09:32:27.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:27.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.84154742199999, 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:27.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.837862798, 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.836057708, 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:27.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:27.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.83551103299999, 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:27.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:27.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.83474258999999, 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:27.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:27.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.834332222, 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.287 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:27.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:27.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.834193046, 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:27.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.296 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.82641423499999, 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:27.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:27.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.82405022099999, 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:27.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.307 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.813298328, 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.308 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:27.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:27.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.813101937, 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:27.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.811607517, 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:27.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:27.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.11136465600001, 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.324 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:27.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:27.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.11134223799999, 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:27.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:27.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.11131909700001, 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:27.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 134.11129413800001, 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:27.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 124.067942624, 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:27.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:27.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:27.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.451464582, 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:27.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:27.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.45136529200001, 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:27.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:27.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:27.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:27.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:27.359 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:27.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:27.361 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:27.361 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:27.357+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:27.362 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:27.363 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:27.363 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:27.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-12T09:32:27.364 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:27.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:27.372 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:27.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:27.389 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:27.406 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:27.406 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:27.425 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:27.443 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:27.461 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:27.462 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:27.459+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:27.481 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:27.484 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:27.485 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:27.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:27.517 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:27.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-12T09:32:27.535 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:27.535 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:27.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:27.596 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:27.593+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:27.663 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:27.659+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:27.670 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:27.670 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:27.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:27.726 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:27.727 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:27.730 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:27.730 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:27.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-12T09:32:27.777 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:27.773+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:27.855 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:27.855 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:27.864 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:27.861+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:27.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:27.945 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:27.946 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:27.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-12T09:32:27.947 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:32:27.974 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:27.971+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:28.062 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.062 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.065 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:28.063+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:28.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:28.113 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:28.113 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:28.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:28.117 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:28.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-12T09:32:28.205 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.205 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.208 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:28.207+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:28.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:28.267 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:28.267+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:32:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:32:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:32:28.326 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:32:28.326 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:28.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-12T09:32:28.341 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.341 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:28.368 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:28.365+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:28.468 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:28.465+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:28.488 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.489 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:28.512 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:28.515 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:28.515 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:32:28.569 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:28.565+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:28.619 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.619 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:28.680 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:28.679+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:28.761 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.761 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.771 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:28.765+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:28.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:28.872 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:28.869+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:28.953 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:28.953 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:28.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:28.973 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:28.969+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:29.073 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:29.071+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:29.109 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:29.109 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:29.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:29.216 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:29.215+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:29.275 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:29.269+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:29.297 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:29.298 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:29.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:29.399 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:29.399+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:29.476 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:29.475+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:29.481 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:29.481 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:29.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:29.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:29.578 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:29.577+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:29.615 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:29.633 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:29.650 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:29.650 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:29.658 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:29.658 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:29.666 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:29.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:29.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:29.688 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:29.687+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:29.708 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:29.708 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:29.729 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:29.750 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:29.750 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:29.754 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:29.755 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:29.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.357803712000006, 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.357778480999997, 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.357740561, 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.357710560000001, 2024-09-12T09:32:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.349676138999996, 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:29.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.349355266000003, 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.348879960999994, 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.348851213000003, 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.348300746999996, 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.348257924999999, 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.348212033999999, 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:29.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.347700476, 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:29.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.347652697000001, 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.858 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.347597050999994, 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:29.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:29.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.347565029999998, 2024-09-12T09:32:29.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.347000881, 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.346606980000004, 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.346215282000003, 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.978349163999994, 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:29.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:29.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:29.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:29.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.119872368000003, 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:29.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:29.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:29.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:29.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:29.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:29.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:29.898 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:29.898 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:29.791+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:29.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:29.944 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:29.943+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:29.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:32:30.001 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:29.997+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:30.041 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:30.041 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:30.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:30.103 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:30.099+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:30.203 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:30.197+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:30.228 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:30.228 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:30.228 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:30.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:30.304 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:30.303+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:30.405 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:30.401+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.11708937500001, 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 141.65759314600001, 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58792 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13008896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28417619499999, 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069822236999999995, 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58792 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121965+0000", 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121966+0000", 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121971+0000", 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.87e-06 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121985+0000", 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.121995+0000", 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0081999999999999e-05 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191468+0000", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069473573999999996 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191532+0000", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3859e-05 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191787+0000", 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025499399999999998 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28412894799999, 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.283835872, 2024-09-12T09:32:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28327393999999, 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.283222577, 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28264969700001, 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.282497398, 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28214030800001, 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28151533799999, 2024-09-12T09:32:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28146400200001, 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28066921199999, 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28059432200001, 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.280542002, 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:30.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.280493404, 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:30.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:30.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.500 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.28023893899999, 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:30.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.27678464900001, 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:30.510 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.219216102, 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.511 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.74794486499999, 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:30.524 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:30.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:30.526 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:30.525+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:30.627 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:30.627+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:30.698 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:30.698 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:30.698 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:30.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:30.756 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:30.753+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:30.829 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:30.827+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:30.834 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:30.834 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:30.835 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:30.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:30.936 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:30.933+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:32:31.030 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:31.025+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61944036099999, 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61934509899999, 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.615660475, 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61385538499999, 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61330871000001, 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61254026699999, 2024-09-12T09:32:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.612129899, 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.61199072299999, 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.60421191200001, 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.60184789799999, 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.591096005, 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.59089961399999, 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.58940519399999, 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.108 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:31.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:31.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:31.113 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.889162333, 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:31.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:31.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.88913991499999, 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:31.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:31.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.889116774, 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:31.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 137.889091815, 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:31.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:31.134 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.84574030100001, 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:31.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:31.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:31.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:31.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:31.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.229262259, 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:31.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:31.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.229162969, 2024-09-12T09:32:31.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:31.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:31.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.153 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:31.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:31.155 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:31.151+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:31.255 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:31.255+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:31.333 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:31.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:31.333 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:31.333 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:31.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:31.392 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:31.389+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:31.458 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:31.455+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:31.469 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:31.469 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:31.470 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:31.470 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:31.470 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:31.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:31.559 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:31.555+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899750101999999, 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:31.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:31.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:31.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899699288999997, 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:31.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899662624999998, 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:31.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:31.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899603292000002, 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:31.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:31.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:31.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:31.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899573283999999, 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:31.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899554449, 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:31.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:31.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:31.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899527730999999, 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:31.689 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:31.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:31.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899506965, 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:31.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:31.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:31.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.89948725, 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:31.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:31.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899457054000003, 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899426486000003, 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:31.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899407683, 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:31.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899358589000002, 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:31.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:31.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899324094000001, 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899291316000003, 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:31.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:31.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899256184999999, 2024-09-12T09:32:31.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:31.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:31.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899213861, 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:31.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899178354, 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899144084, 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:31.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.899090747000002, 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:31.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:31.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:31.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:31.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:31.755 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:31.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:31.757 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:31.755+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:31.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:31.857 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:31.855+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:31.885 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:31.885 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:31.885 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:31.885 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:31.898 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:31.900 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:31.916 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:31.916 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:31.933 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:31.951 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:31.958 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:31.953+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:31.969 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:31.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:32.015 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:32.029 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:32.029 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:32.029 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:32.029 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:32.029 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:32.037 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:32.037 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:32.041 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:32.041 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:32.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:32.083 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:32.083+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:32.159 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.153+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.214 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.215 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:32.276 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.273+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.361 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:32.359+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:32.367 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.367 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:32.462 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.461+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.549 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.549 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.563 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.561+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:32.664 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.661+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.683 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.683 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:32.764 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:32.761+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:32.817 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.817 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:32.865 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:32.861+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:32.951 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:32.951 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:32.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:32.966 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:32.963+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:33.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:32:33.066 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:33.061+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:33.143 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.144 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:33.166 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:33.161+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:33.267 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:33.261+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:33.287 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.287 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:33.368 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:33.365+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:33.420 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.421 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:33.469 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:33.465+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:33.516 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:33.516 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:33.517 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-12T09:32:33.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:32:33.564 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.564 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.569 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:33.567+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:33.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:33.670 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:33.665+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:33.712 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.712 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:33.812 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:33.809+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 175, 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:32:32.985191+0000", 2024-09-12T09:32:33.831 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:31:57.668482+0000", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:51.831699+0000", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:32:33.832 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:32:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:33.834 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:33.835 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:33.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.837 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.838 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 7.690000057220459, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 7.690000057220459, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.839 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:33.840 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.844 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:33.846 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:33.847 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "175", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 55, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 175, 2024-09-12T09:32:33.848 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:33.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:33.850 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2400000095367432, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2400000095367432, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:33.851 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 140, 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 141, 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:32:33.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:32:33.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:32:33.857 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.858 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 141, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.863 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.864 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 138, 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:32:33.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:32:33.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:32:33.874 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.875 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.876 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 164, 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 155, 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:32:33.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:32:33.887 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.0386199951171875, 2024-09-12T09:32:33.888 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 154, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:33.894 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 164, 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:32:33.900 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 141 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:33.901 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.902 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.904 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:33.905 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:32:33.906 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.907 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1a", 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.909 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1", 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.3", 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.10", 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.17", 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:32:33.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 30, 2024-09-12T09:32:33.914 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 26 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 30, 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:33.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:33.916 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:33.918 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:33.918 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:33.919 INFO:tasks.thrashosds.thrasher:j is {'epoch': 175, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:32:32.985191+0000', 'last_up_change': '2024-09-12T09:31:57.668482+0000', 'last_in_change': '2024-09-12T09:31:51.831699+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 175, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 141, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0386199951171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1', 'osds': [4, 3]}, {'pgid': '3.3', 'osds': [4, 6]}, {'pgid': '3.b', 'osds': [3, 4]}, {'pgid': '3.10', 'osds': [6, 5]}, {'pgid': '3.17', 'osds': [5, 3]}, {'pgid': '3.18', 'osds': [3, 1]}, {'pgid': '3.1b', 'osds': [4, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 26}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:32:33.920 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1a 2024-09-12T09:32:33.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.1a 2024-09-12T09:32:33.920 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:33.915+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:33.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:32:34.018 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:34.015+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:34.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:34.101 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:34.117 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:34.119 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:34.115+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:34.205 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:34.219 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:34.213+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:34.223 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:34.241 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:34.241 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:34.242 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:34.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:34.299 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:34.320 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:34.319+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:34.335 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:34.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:34.339 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:34.421 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:34.417+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:34.422 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:34.422 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:34.422 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:34.422 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:34.422 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:34.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:34.524 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:34.521+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.19303376600001, 2024-09-12T09:32:34.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:34.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.19293850400001, 2024-09-12T09:32:34.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:34.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:34.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:34.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18925388, 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:34.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18744878999999, 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:34.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:34.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18690211500001, 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:34.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18613367200001, 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:34.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18572330399999, 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:34.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.18558412799999, 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.643 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:34.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.17780531700001, 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:34.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.17544130300001, 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:34.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:34.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:34.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.16468940999999, 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:34.661 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:34.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:34.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.16449301899999, 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:34.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:34.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.16299859899999, 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:34.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 141.462755738, 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:34.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:34.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 141.46273332000001, 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:34.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:34.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 141.462710179, 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 141.46268522, 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 131.419333706, 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:34.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:34.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 130.80285566399999, 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:34.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 130.80275637400001, 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:34.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:34.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:34.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:34.716 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:34.718 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:34.718 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:34.715+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:34.819 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:34.819+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:34.885 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:34.885 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:34.885 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:34.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:34.920 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:34.919+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:clear 3.1a pg_upmap_items mapping 2024-09-12T09:32:35.016 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:35.016 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:35.016 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:35.016 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:35.016 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:35.022 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:35.019+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:35.040 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:35.122 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:35.119+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:35.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.40773428, 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407683466999998, 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:35.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:35.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.176 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:35.176 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407646802999999, 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:35.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407587470000003, 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:35.181 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:35.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:35.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407557462, 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:35.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407538627000001, 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407511909, 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:35.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:35.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407491143000001, 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:35.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407471428000001, 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:35.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407441231999996, 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:35.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407410663999997, 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:35.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.219 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.219 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407391861000001, 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:35.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407342767000003, 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:35.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407308272000002, 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:35.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407275493999997, 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:35.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407240363, 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.238 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:35.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:35.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407198039000001, 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.247 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407162532000001, 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:35.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:35.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407128262000001, 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.252 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:35.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:35.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.407074925000003, 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:35.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:35.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:35.263 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:35.259+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:35.364 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:35.363+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:35.390 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:35.390 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:35.390 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:35.405 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:35.465 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:35.461+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:35.525 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:35.526 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:35.526 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:35.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:35.566 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:35.563+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:35.667 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:35.667+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.494787102000004, 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.492663484000005, 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.492643657000002, 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.490548623999999, 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488514017, 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488475300000005, 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:35.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:35.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:35.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488438316, 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:35.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:35.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488423932000003, 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:35.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488402941000004, 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:35.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488391469999996, 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:35.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.488365985000002, 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:35.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.486178705, 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:35.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.486168336999995, 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:35.778 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:35.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:35.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.486142371, 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:35.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:35.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:35.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484450549000002, 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.794 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484435993000005, 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:35.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:35.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484415623000004, 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:35.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:35.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484376995000005, 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:35.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484362472000001, 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:35.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:35.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.484352056999995, 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:35.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:35.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:35.821 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:35.823 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:35.817+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:35.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:35.924 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:35.921+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:35.960 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:35.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:35.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:36.025 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:36.023+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:36.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:32:36.101 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:36.101 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:36.101 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:36.101 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:36.101 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:36.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:36.159 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:36.159+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:36.227 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:36.221+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.04220914800001, 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:32:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:32:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.02723841900001, 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:32:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.710391415, 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:32:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.71034505599999, 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:32:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.710237836, 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:32:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:32:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.710144345, 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:32:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.710077832, 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:32:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.71001702300001, 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:32:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:32:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709957731, 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:32:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709897222, 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.70981916700001, 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:32:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:32:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.70973379199999, 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:32:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709665473, 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:32:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709572888, 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:32:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709516979, 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:32:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709484656, 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:32:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709451063, 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:32:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709400121, 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709301966, 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:32:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:32:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.709244872, 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:32:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:36.342 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:36.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:36.344 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:36.341+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:36.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:36.444 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:36.443+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:36.518 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:36.518 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:36.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:36.533 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:36.533 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:36.546 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:36.541+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:36.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:36.567 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:36.587 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:36.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:36.629 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:36.629 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:36.647 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:36.641+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:36.651 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:36.654 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:36.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:36.747 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:36.741+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:36.843 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:36.843 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:36.848 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:36.847+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:36.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:36.950 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:36.947+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:37.021 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.021 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:32:37.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:37.050 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:37.045+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:37.151 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:37.151+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:37.221 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.221 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:37.252 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:37.249+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:37.353 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:37.351+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:37.405 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.405 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:37.464 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:37.463+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:37.550 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.550 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.554 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:37.551+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:37.564 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:37.655 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:37.653+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:37.692 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.692 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:37.757 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:37.755+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:37.857 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:37.853+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:37.887 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:37.887 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:37.903 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:37.988 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:37.985+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:38.059 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:38.053+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:38.089 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:38.169 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:38.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:38.172 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:38.165+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:38.271 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:38.265+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:38.346 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:38.346 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:38.360 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:38.404 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:38.401+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:38.473 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:38.469+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:38.528 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:38.529 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:38.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:38.573 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:38.571+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:38.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:38.674 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:38.669+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:38.706 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:38.706 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:38.720 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:38.775 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:38.775+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:38.816 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:38.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:38.849 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:38.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:38.870 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:38.870 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:38.876 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:38.875+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:38.889 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:38.889 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:38.891 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:38.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:38.912 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:38.913 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:38.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:38.932 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:38.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:38.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:38.980 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:38.979+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:39.031 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:39.031 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:39.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.274058107000002, 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:39.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.274007294, 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:39.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.041 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273970630000001, 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:39.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:39.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:39.046 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273911296999998, 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:39.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:39.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:39.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273881289000002, 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:39.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273862454000003, 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:39.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273835736000002, 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273814969999997, 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273795255000003, 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273765058999999, 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273734490999999, 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273715688000003, 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273666593999998, 2024-09-12T09:32:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273632098999997, 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273599320999999, 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273564190000002, 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273521866000003, 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273486359000003, 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273452089000003, 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.273398751999999, 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.130 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:39.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:32:39.133 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:39.133 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:39.129+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:39.233 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:39.231+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:39.315 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:39.315 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:39.315 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:39.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:39.334 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:39.331+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:39.435 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:39.429+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:39.453 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:39.454 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:39.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:39.556 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:39.553+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.421630829999998, 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.419507211999999, 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.419487384999996, 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.417392351999993, 2024-09-12T09:32:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415357744999994, 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415319027999999, 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:39.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:39.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415282043999994, 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:39.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:39.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:39.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415267659999998, 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:39.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:39.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415246668999998, 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:39.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415235198000005, 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:39.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:39.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:39.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.415209712999996, 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:39.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.413022432999995, 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:39.713 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:39.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:39.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.413012065000004, 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:39.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:39.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.412986098999994, 2024-09-12T09:32:39.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:39.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:39.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:39.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411294276999996, 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:39.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:39.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:39.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411279721, 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:39.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411259350999998, 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:39.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:39.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:39.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411220723, 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:39.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:39.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411206199999995, 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.759 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:39.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:39.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:39.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:39.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.411195785000004, 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:39.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:39.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:39.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:39.772 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:39.773 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:39.635+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:39.773 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:39.769+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:39.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:39.874 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:39.869+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:39.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:39.975 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:39.971+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:40.042 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.042 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:40.043 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:40.043 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.049 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:40.052 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:40.052 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:32:40.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:32:40.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:32:40.079 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:40.077+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:40.177 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:40.173+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.00775341400001, 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:40.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:32:40.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 136.99278268500001, 2024-09-12T09:32:40.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:32:40.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:40.220 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675935681, 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:32:40.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:32:40.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.225 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675889322, 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.226 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:32:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675782102, 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:32:40.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675688611, 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:32:40.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:40.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.67562209800001, 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:32:40.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675561289, 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:32:40.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675501997, 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:32:40.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675441488, 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:32:40.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:40.262 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675363433, 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:32:40.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675278058, 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:32:40.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675209739, 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:32:40.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.67511715400001, 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:32:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.67506124499999, 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:32:40.283 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:32:40.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:32:40.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.675028922, 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:32:40.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.674995329, 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:32:40.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.674944387, 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.67484623199999, 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:32:40.305 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:40.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:32:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.67478913799999, 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:32:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:32:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.315 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:40.317 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:40.313+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 182, 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:32:40.060492+0000", 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:31:57.668482+0000", 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:51.831699+0000", 2024-09-12T09:32:40.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:32:40.390 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:32:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:32:40.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:32:40.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:40.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 7.690000057220459, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 7.690000057220459, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-12T09:32:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:32:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:32:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:32:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "182", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 55, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 182, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2400000095367432, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2400000095367432, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 140, 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 141, 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:32:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:32:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 179, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:32:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 138, 2024-09-12T09:32:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:32:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:32:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 180, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 155, 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.0386199951171875, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 154, 2024-09-12T09:32:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:32:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 164, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:32:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 141 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:32:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:32:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1", 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.3", 2024-09-12T09:32:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-12T09:32:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.10", 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.17", 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:32:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 26, 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 30 2024-09-12T09:32:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 26, 2024-09-12T09:32:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:32:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.501 INFO:tasks.thrashosds.thrasher:j is {'epoch': 182, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:32:40.060492+0000', 'last_up_change': '2024-09-12T09:31:57.668482+0000', 'last_in_change': '2024-09-12T09:31:51.831699+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 182, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 179, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 180, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0386199951171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [5, 0]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1', 'osds': [4, 3]}, {'pgid': '3.3', 'osds': [4, 6]}, {'pgid': '3.b', 'osds': [3, 4]}, {'pgid': '3.10', 'osds': [6, 5]}, {'pgid': '3.17', 'osds': [5, 3]}, {'pgid': '3.18', 'osds': [3, 1]}, {'pgid': '3.1b', 'osds': [4, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 30}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:32:40.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:32:40.502 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:40.502 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:40.503 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:40.499+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:40.601 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:40.601+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:40.702 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:40.697+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:40.803 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:40.799+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:40.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.93091752999999, 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:40.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.58338764199999, 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:40.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:40.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.58093373200001, 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:40.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:40.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.57882384499999, 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:40.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.08307720000001, 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:40.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.855302683999994, 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:40.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.855242122000007, 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:40.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.855177845, 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:40.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.855110326000002, 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:40.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:40.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.855005700000007, 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854942167999994, 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:40.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:40.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:40.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854904227999995, 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:40.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854875707999994, 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:40.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854826204999995, 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:40.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854781815999999, 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:40.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854719932999998, 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:40.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.887 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.85467242, 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:40.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:40.892 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854644149999999, 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:40.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854616597000003, 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:40.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.854585814000004, 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:40.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:40.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:40.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:40.906 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:40.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:40.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:32:40.911 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-12T09:32:40.069275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1322255538,"num_objects":1272,"num_object_clones":291,"num_object_copies":2544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":209,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":1272,"num_whiteouts":553,"num_read":248958,"num_read_kb":1577320,"num_write":22107,"num_write_kb":853909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":786529270,"num_keys_recovered":692,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18195,"ondisk_log_size":18195,"up":68,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456184,"kb_used_data":87300,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754518536,"statfs":{"total":773094113280,"available":772626980864,"internally_reserved":0,"allocated":89395200,"data_stored":1046906408,"data_compressed":9886076,"data_compressed_allocated":64884736,"data_compressed_original":1033437308,"omap_allocated":12752,"internal_metadata":377540144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29378398,"num_objects":-86,"num_object_clones":-27,"num_object_copies":-172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":-28,"num_objects_unfound":0,"num_objects_dirty":-86,"num_whiteouts":-8,"num_read":1307,"num_read_kb":1103,"num_write":175,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.045208"},"pg_stats":[{"pgid":"3.1f","version":"174'740","reported_seq":6351,"reported_epoch":177,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:35.028946+0000","last_change":"2024-09-12T09:31:58.689191+0000","last_active":"2024-09-12T09:32:35.028946+0000","last_peered":"2024-09-12T09:32:35.028946+0000","last_clean":"2024-09-12T09:31:51.668701+0000","last_became_active":"2024-09-12T09:31:58.689191+0000","last_became_peered":"2024-09-12T09:31:58.689191+0000","last_unstale":"2024-09-12T09:32:35.028946+0000","last_undegraded":"2024-09-12T09:31:58.686707+0000","last_fullsized":"2024-09-12T09:31:58.686488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":28,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:50.310644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":90342077,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":8213,"num_read_kb":95267,"num_write":944,"num_write_kb":47653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":37755293,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":47}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"177'659","reported_seq":5932,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.028099+0000","last_change":"2024-09-12T09:32:35.028099+0000","last_active":"2024-09-12T09:32:35.028099+0000","last_peered":"2024-09-12T09:32:35.028099+0000","last_clean":"2024-09-12T09:32:35.028099+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:32:35.028099+0000","last_undegraded":"2024-09-12T09:32:35.028099+0000","last_fullsized":"2024-09-12T09:32:35.028099+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:37.543271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002120257,"stat_sum":{"num_bytes":41789349,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":8337,"num_read_kb":40978,"num_write":778,"num_write_kb":38748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.1d","version":"174'482","reported_seq":7932,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.030883+0000","last_change":"2024-09-12T09:32:35.031379+0000","last_active":"2024-09-12T09:32:35.030883+0000","last_peered":"2024-09-12T09:32:35.030883+0000","last_clean":"2024-09-12T09:32:35.030883+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:32:35.030883+0000","last_undegraded":"2024-09-12T09:32:35.030883+0000","last_fullsized":"2024-09-12T09:32:35.030883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:19.888101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029922669999999998,"stat_sum":{"num_bytes":43645672,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":8919,"num_read_kb":78476,"num_write":469,"num_write_kb":43501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.1c","version":"177'671","reported_seq":6736,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.049059+0000","last_change":"2024-09-12T09:32:35.037017+0000","last_active":"2024-09-12T09:32:35.049059+0000","last_peered":"2024-09-12T09:32:35.049059+0000","last_clean":"2024-09-12T09:32:35.049059+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:32:35.049059+0000","last_undegraded":"2024-09-12T09:32:35.049059+0000","last_fullsized":"2024-09-12T09:32:35.049059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:56.364571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082276780000000004,"stat_sum":{"num_bytes":19972439,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":8077,"num_read_kb":52288,"num_write":739,"num_write_kb":16449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.1b","version":"177'334","reported_seq":6267,"reported_epoch":178,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:36.023828+0000","last_change":"2024-09-12T09:32:35.034844+0000","last_active":"2024-09-12T09:32:36.023828+0000","last_peered":"2024-09-12T09:32:36.023828+0000","last_clean":"2024-09-12T09:32:36.023828+0000","last_became_active":"2024-09-12T09:31:59.687414+0000","last_became_peered":"2024-09-12T09:31:59.687414+0000","last_unstale":"2024-09-12T09:32:36.023828+0000","last_undegraded":"2024-09-12T09:32:36.023828+0000","last_fullsized":"2024-09-12T09:32:36.023828+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:32.814910+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014937964999999999,"stat_sum":{"num_bytes":25045554,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":33,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":5983,"num_read_kb":20622,"num_write":330,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[{"shards":"4,7","objects":33}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.1a","version":"173'538","reported_seq":9051,"reported_epoch":178,"state":"remapped+peering","last_fresh":"2024-09-12T09:32:36.020835+0000","last_change":"2024-09-12T09:32:36.020835+0000","last_active":"2024-09-12T09:32:36.020835+0000","last_peered":"2024-09-12T09:32:31.136003+0000","last_clean":"2024-09-12T09:32:31.136003+0000","last_became_active":"2024-09-12T09:31:43.679805+0000","last_became_peered":"2024-09-12T09:31:43.679805+0000","last_unstale":"2024-09-12T09:32:36.020835+0000","last_undegraded":"2024-09-12T09:32:36.020835+0000","last_fullsized":"2024-09-12T09:32:36.020835+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0086961079999999993,"stat_sum":{"num_bytes":9197978,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":8716,"num_read_kb":31786,"num_write":583,"num_write_kb":38634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":10753735,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"1d","length":"1b"}]},{"pgid":"3.19","version":"177'399","reported_seq":3719,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.020290+0000","last_change":"2024-09-12T09:32:35.029486+0000","last_active":"2024-09-12T09:32:36.020290+0000","last_peered":"2024-09-12T09:32:36.020290+0000","last_clean":"2024-09-12T09:32:36.020290+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:32:36.020290+0000","last_undegraded":"2024-09-12T09:32:36.020290+0000","last_fullsized":"2024-09-12T09:32:36.020290+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:27.410111+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0077290969999999999,"stat_sum":{"num_bytes":24234961,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":4027,"num_read_kb":15034,"num_write":453,"num_write_kb":6310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.18","version":"177'442","reported_seq":4021,"reported_epoch":177,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:35.032551+0000","last_change":"2024-09-12T09:32:35.033509+0000","last_active":"2024-09-12T09:32:35.032551+0000","last_peered":"2024-09-12T09:32:35.032551+0000","last_clean":"2024-09-12T09:32:35.032551+0000","last_became_active":"2024-09-12T09:31:59.685845+0000","last_became_peered":"2024-09-12T09:31:59.685845+0000","last_unstale":"2024-09-12T09:32:35.032551+0000","last_undegraded":"2024-09-12T09:32:35.032551+0000","last_fullsized":"2024-09-12T09:32:35.032551+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:53.569066+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0077122509999999998,"stat_sum":{"num_bytes":9334507,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":34,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":2601,"num_read_kb":10167,"num_write":474,"num_write_kb":8063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":34}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.17","version":"176'601","reported_seq":4890,"reported_epoch":177,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:35.037909+0000","last_change":"2024-09-12T09:32:35.038324+0000","last_active":"2024-09-12T09:32:35.037909+0000","last_peered":"2024-09-12T09:32:35.037909+0000","last_clean":"2024-09-12T09:32:35.037909+0000","last_became_active":"2024-09-12T09:31:59.686284+0000","last_became_peered":"2024-09-12T09:31:59.686284+0000","last_unstale":"2024-09-12T09:32:35.037909+0000","last_undegraded":"2024-09-12T09:32:35.037909+0000","last_fullsized":"2024-09-12T09:32:35.037909+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:06.877071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0090099220000000001,"stat_sum":{"num_bytes":42464978,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":6896,"num_read_kb":53657,"num_write":709,"num_write_kb":39724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":27}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.16","version":"177'511","reported_seq":15723,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.034952+0000","last_change":"2024-09-12T09:32:35.035352+0000","last_active":"2024-09-12T09:32:35.034952+0000","last_peered":"2024-09-12T09:32:35.034952+0000","last_clean":"2024-09-12T09:32:35.034952+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:32:35.034952+0000","last_undegraded":"2024-09-12T09:32:35.034952+0000","last_fullsized":"2024-09-12T09:32:35.034952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:39.727198+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0067664120000000003,"stat_sum":{"num_bytes":62654734,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":15916,"num_read_kb":96835,"num_write":562,"num_write_kb":40304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.15","version":"177'769","reported_seq":8589,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.026351+0000","last_change":"2024-09-12T09:32:35.031111+0000","last_active":"2024-09-12T09:32:37.026351+0000","last_peered":"2024-09-12T09:32:37.026351+0000","last_clean":"2024-09-12T09:32:37.026351+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:32:37.026351+0000","last_undegraded":"2024-09-12T09:32:37.026351+0000","last_fullsized":"2024-09-12T09:32:37.026351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:38.306130+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0093778300000000002,"stat_sum":{"num_bytes":82189578,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":9147,"num_read_kb":102680,"num_write":999,"num_write_kb":47722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.14","version":"177'774","reported_seq":12076,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.023943+0000","last_change":"2024-09-12T09:32:35.044891+0000","last_active":"2024-09-12T09:32:36.023943+0000","last_peered":"2024-09-12T09:32:36.023943+0000","last_clean":"2024-09-12T09:32:36.023943+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:32:36.023943+0000","last_undegraded":"2024-09-12T09:32:36.023943+0000","last_fullsized":"2024-09-12T09:32:36.023943+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:58:25.935725+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.023941289000000001,"stat_sum":{"num_bytes":21756971,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":13089,"num_read_kb":53766,"num_write":1098,"num_write_kb":31328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.025662+0000","last_change":"2024-09-12T09:32:22.911749+0000","last_active":"2024-09-12T09:32:37.025662+0000","last_peered":"2024-09-12T09:32:37.025662+0000","last_clean":"2024-09-12T09:32:37.025662+0000","last_became_active":"2024-09-12T09:32:22.911415+0000","last_became_peered":"2024-09-12T09:32:22.911415+0000","last_unstale":"2024-09-12T09:32:37.025662+0000","last_undegraded":"2024-09-12T09:32:37.025662+0000","last_fullsized":"2024-09-12T09:32:37.025662+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:37.113174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"176'330","reported_seq":6826,"reported_epoch":178,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:36.021533+0000","last_change":"2024-09-12T09:31:58.680982+0000","last_active":"2024-09-12T09:32:36.021533+0000","last_peered":"2024-09-12T09:32:36.021533+0000","last_clean":"2024-09-12T09:31:51.383028+0000","last_became_active":"2024-09-12T09:31:58.680982+0000","last_became_peered":"2024-09-12T09:31:58.680982+0000","last_unstale":"2024-09-12T09:32:36.021533+0000","last_undegraded":"2024-09-12T09:31:58.679925+0000","last_fullsized":"2024-09-12T09:31:58.679826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":28,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:51.494162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":56581998,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":6669,"num_read_kb":29466,"num_write":289,"num_write_kb":10835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":46}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":117,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.020375+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:32:36.020375+0000","last_peered":"2024-09-12T09:32:36.020375+0000","last_clean":"2024-09-12T09:32:36.020375+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:32:36.020375+0000","last_undegraded":"2024-09-12T09:32:36.020375+0000","last_fullsized":"2024-09-12T09:32:36.020375+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"174'401","reported_seq":3954,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.025587+0000","last_change":"2024-09-12T09:32:35.025658+0000","last_active":"2024-09-12T09:32:37.025587+0000","last_peered":"2024-09-12T09:32:37.025587+0000","last_clean":"2024-09-12T09:32:37.025587+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:32:37.025587+0000","last_undegraded":"2024-09-12T09:32:37.025587+0000","last_fullsized":"2024-09-12T09:32:37.025587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:53.077652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035144659999999999,"stat_sum":{"num_bytes":25079818,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":3901,"num_read_kb":30598,"num_write":444,"num_write_kb":26827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.5","version":"0'0","reported_seq":245,"reported_epoch":179,"state":"active+undersized","last_fresh":"2024-09-12T09:32:37.026393+0000","last_change":"2024-09-12T09:31:58.689633+0000","last_active":"2024-09-12T09:32:37.026393+0000","last_peered":"2024-09-12T09:32:37.026393+0000","last_clean":"2024-09-12T09:31:56.850902+0000","last_became_active":"2024-09-12T09:31:58.689633+0000","last_became_peered":"2024-09-12T09:31:58.689633+0000","last_unstale":"2024-09-12T09:32:37.026393+0000","last_undegraded":"2024-09-12T09:32:37.026393+0000","last_fullsized":"2024-09-12T09:31:58.684053+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:23.955347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"176'1116","reported_seq":7958,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.021169+0000","last_change":"2024-09-12T09:32:35.030388+0000","last_active":"2024-09-12T09:32:36.021169+0000","last_peered":"2024-09-12T09:32:36.021169+0000","last_clean":"2024-09-12T09:32:36.021169+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:32:36.021169+0000","last_undegraded":"2024-09-12T09:32:36.021169+0000","last_fullsized":"2024-09-12T09:32:36.021169+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:11.375497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007422283,"stat_sum":{"num_bytes":23545841,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":11145,"num_read_kb":45471,"num_write":1735,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.4","version":"0'0","reported_seq":126,"reported_epoch":178,"state":"active+undersized","last_fresh":"2024-09-12T09:32:36.021338+0000","last_change":"2024-09-12T09:31:58.680941+0000","last_active":"2024-09-12T09:32:36.021338+0000","last_peered":"2024-09-12T09:32:36.021338+0000","last_clean":"2024-09-12T09:31:56.702759+0000","last_became_active":"2024-09-12T09:31:58.680941+0000","last_became_peered":"2024-09-12T09:31:58.680941+0000","last_unstale":"2024-09-12T09:32:36.021338+0000","last_undegraded":"2024-09-12T09:32:36.021338+0000","last_fullsized":"2024-09-12T09:31:58.679822+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:32.828980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"177'251","reported_seq":13979,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.038756+0000","last_change":"2024-09-12T09:32:35.039065+0000","last_active":"2024-09-12T09:32:35.038756+0000","last_peered":"2024-09-12T09:32:35.038756+0000","last_clean":"2024-09-12T09:32:35.038756+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:32:35.038756+0000","last_undegraded":"2024-09-12T09:32:35.038756+0000","last_fullsized":"2024-09-12T09:32:35.038756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:00.003244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010471362,"stat_sum":{"num_bytes":34066660,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":13557,"num_read_kb":60534,"num_write":211,"num_write_kb":14069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.3","version":"177'1025","reported_seq":6050,"reported_epoch":178,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:36.023851+0000","last_change":"2024-09-12T09:32:35.049000+0000","last_active":"2024-09-12T09:32:36.023851+0000","last_peered":"2024-09-12T09:32:36.023851+0000","last_clean":"2024-09-12T09:32:36.023851+0000","last_became_active":"2024-09-12T09:31:59.688563+0000","last_became_peered":"2024-09-12T09:31:59.688563+0000","last_unstale":"2024-09-12T09:32:36.023851+0000","last_undegraded":"2024-09-12T09:32:36.023851+0000","last_fullsized":"2024-09-12T09:32:36.023851+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:47:20.682946+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.028833573000000001,"stat_sum":{"num_bytes":22181904,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":7740,"num_read_kb":25033,"num_write":1397,"num_write_kb":11269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":46}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.2","version":"21'2","reported_seq":374,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.028448+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:32:35.028448+0000","last_peered":"2024-09-12T09:32:35.028448+0000","last_clean":"2024-09-12T09:32:35.028448+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:32:35.028448+0000","last_undegraded":"2024-09-12T09:32:35.028448+0000","last_fullsized":"2024-09-12T09:32:35.028448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"177'374","reported_seq":4985,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.020380+0000","last_change":"2024-09-12T09:32:35.031237+0000","last_active":"2024-09-12T09:32:36.020380+0000","last_peered":"2024-09-12T09:32:36.020380+0000","last_clean":"2024-09-12T09:32:36.020380+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:32:36.020380+0000","last_undegraded":"2024-09-12T09:32:36.020380+0000","last_fullsized":"2024-09-12T09:32:36.020380+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:08.773577+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0091751880000000008,"stat_sum":{"num_bytes":43403017,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":4684,"num_read_kb":38015,"num_write":329,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.1","version":"0'0","reported_seq":335,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.022962+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:32:36.022962+0000","last_peered":"2024-09-12T09:32:36.022962+0000","last_clean":"2024-09-12T09:32:36.022962+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:32:36.022962+0000","last_undegraded":"2024-09-12T09:32:36.022962+0000","last_fullsized":"2024-09-12T09:32:36.022962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"177'487","reported_seq":13142,"reported_epoch":178,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:36.023450+0000","last_change":"2024-09-12T09:32:35.029899+0000","last_active":"2024-09-12T09:32:36.023450+0000","last_peered":"2024-09-12T09:32:36.023450+0000","last_clean":"2024-09-12T09:32:36.023450+0000","last_became_active":"2024-09-12T09:31:59.684097+0000","last_became_peered":"2024-09-12T09:31:59.684097+0000","last_unstale":"2024-09-12T09:32:36.023450+0000","last_undegraded":"2024-09-12T09:32:36.023450+0000","last_fullsized":"2024-09-12T09:32:36.023450+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:48:27.517191+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.010251573999999999,"stat_sum":{"num_bytes":36887401,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":28,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":13820,"num_read_kb":38989,"num_write":515,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":10010383,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":28}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.0","version":"0'0","reported_seq":248,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.025905+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:32:37.025905+0000","last_peered":"2024-09-12T09:32:37.025905+0000","last_clean":"2024-09-12T09:32:37.025905+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:32:37.025905+0000","last_undegraded":"2024-09-12T09:32:37.025905+0000","last_fullsized":"2024-09-12T09:32:37.025905+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"177'350","reported_seq":5513,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.034745+0000","last_change":"2024-09-12T09:32:35.035106+0000","last_active":"2024-09-12T09:32:35.034745+0000","last_peered":"2024-09-12T09:32:35.034745+0000","last_clean":"2024-09-12T09:32:35.034745+0000","last_became_active":"2024-09-12T09:32:13.696998+0000","last_became_peered":"2024-09-12T09:32:13.696998+0000","last_unstale":"2024-09-12T09:32:35.034745+0000","last_undegraded":"2024-09-12T09:32:35.034745+0000","last_fullsized":"2024-09-12T09:32:35.034745+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:19.812435+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0068845870000000002,"stat_sum":{"num_bytes":30526904,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":5938,"num_read_kb":46699,"num_write":348,"num_write_kb":14929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":9357778,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"2.3","version":"19'1","reported_seq":333,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.028234+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:32:35.028234+0000","last_peered":"2024-09-12T09:32:35.028234+0000","last_clean":"2024-09-12T09:32:35.028234+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:32:35.028234+0000","last_undegraded":"2024-09-12T09:32:35.028234+0000","last_fullsized":"2024-09-12T09:32:35.028234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":275,"reported_epoch":179,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:37.025299+0000","last_change":"2024-09-12T09:31:58.683002+0000","last_active":"2024-09-12T09:32:37.025299+0000","last_peered":"2024-09-12T09:32:37.025299+0000","last_clean":"2024-09-12T09:31:56.850615+0000","last_became_active":"2024-09-12T09:31:58.683002+0000","last_became_peered":"2024-09-12T09:31:58.683002+0000","last_unstale":"2024-09-12T09:32:37.025299+0000","last_undegraded":"2024-09-12T09:31:58.681286+0000","last_fullsized":"2024-09-12T09:31:58.681061+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:41.462657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"174'1250","reported_seq":6101,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.028345+0000","last_change":"2024-09-12T09:32:35.029446+0000","last_active":"2024-09-12T09:32:35.028345+0000","last_peered":"2024-09-12T09:32:35.028345+0000","last_clean":"2024-09-12T09:32:35.028345+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:32:35.028345+0000","last_undegraded":"2024-09-12T09:32:35.028345+0000","last_fullsized":"2024-09-12T09:32:35.028345+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1250,"log_dups_size":0,"ondisk_log_size":1250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:30:24.185383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00282139,"stat_sum":{"num_bytes":38826208,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":10942,"num_read_kb":56793,"num_write":1899,"num_write_kb":24766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.9","version":"173'697","reported_seq":4893,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.023560+0000","last_change":"2024-09-12T09:32:35.023272+0000","last_active":"2024-09-12T09:32:36.023560+0000","last_peered":"2024-09-12T09:32:36.023560+0000","last_clean":"2024-09-12T09:32:36.023560+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:32:36.023560+0000","last_undegraded":"2024-09-12T09:32:36.023560+0000","last_fullsized":"2024-09-12T09:32:36.023560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:06:19.723154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036067870000000002,"stat_sum":{"num_bytes":22133403,"num_objects":33,"num_object_clones":4,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":6020,"num_read_kb":29995,"num_write":895,"num_write_kb":15061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.a","version":"176'430","reported_seq":5647,"reported_epoch":178,"state":"active+clean","last_fresh":"2024-09-12T09:32:36.024962+0000","last_change":"2024-09-12T09:32:35.041271+0000","last_active":"2024-09-12T09:32:36.024962+0000","last_peered":"2024-09-12T09:32:36.024962+0000","last_clean":"2024-09-12T09:32:36.024962+0000","last_became_active":"2024-09-12T09:32:22.909677+0000","last_became_peered":"2024-09-12T09:32:22.909677+0000","last_unstale":"2024-09-12T09:32:36.024962+0000","last_undegraded":"2024-09-12T09:32:36.024962+0000","last_fullsized":"2024-09-12T09:32:36.024962+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:53:27.966151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020635259999999999,"stat_sum":{"num_bytes":28661649,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":4991,"num_read_kb":17242,"num_write":498,"num_write_kb":10436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.b","version":"177'465","reported_seq":6535,"reported_epoch":177,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:35.040245+0000","last_change":"2024-09-12T09:32:35.040710+0000","last_active":"2024-09-12T09:32:35.040245+0000","last_peered":"2024-09-12T09:32:35.040245+0000","last_clean":"2024-09-12T09:32:35.040245+0000","last_became_active":"2024-09-12T09:31:59.685428+0000","last_became_peered":"2024-09-12T09:31:59.685428+0000","last_unstale":"2024-09-12T09:32:35.040245+0000","last_undegraded":"2024-09-12T09:32:35.040245+0000","last_fullsized":"2024-09-12T09:32:35.040245+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:38.637632+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014242846999999999,"stat_sum":{"num_bytes":72723981,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":43,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":6709,"num_read_kb":72316,"num_write":618,"num_write_kb":32938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":43}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.c","version":"173'433","reported_seq":4107,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-12T09:32:35.037085+0000","last_change":"2024-09-12T09:32:35.037768+0000","last_active":"2024-09-12T09:32:35.037085+0000","last_peered":"2024-09-12T09:32:35.037085+0000","last_clean":"2024-09-12T09:32:35.037085+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:32:35.037085+0000","last_undegraded":"2024-09-12T09:32:35.037085+0000","last_fullsized":"2024-09-12T09:32:35.037085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:30:54.621456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084749890000000005,"stat_sum":{"num_bytes":26022144,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":5163,"num_read_kb":44355,"num_write":418,"num_write_kb":19905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.d","version":"176'537","reported_seq":8847,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.026054+0000","last_change":"2024-09-12T09:32:35.023687+0000","last_active":"2024-09-12T09:32:37.026054+0000","last_peered":"2024-09-12T09:32:37.026054+0000","last_clean":"2024-09-12T09:32:37.026054+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:32:37.026054+0000","last_undegraded":"2024-09-12T09:32:37.026054+0000","last_fullsized":"2024-09-12T09:32:37.026054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:34:20.258732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023983250000000002,"stat_sum":{"num_bytes":22812472,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":9501,"num_read_kb":28892,"num_write":575,"num_write_kb":7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.e","version":"176'528","reported_seq":5804,"reported_epoch":177,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:35.029963+0000","last_change":"2024-09-12T09:31:58.688892+0000","last_active":"2024-09-12T09:32:35.029963+0000","last_peered":"2024-09-12T09:32:35.029963+0000","last_clean":"2024-09-12T09:31:56.715994+0000","last_became_active":"2024-09-12T09:31:58.688892+0000","last_became_peered":"2024-09-12T09:31:58.688892+0000","last_unstale":"2024-09-12T09:32:35.029963+0000","last_undegraded":"2024-09-12T09:31:58.686246+0000","last_fullsized":"2024-09-12T09:31:58.686019+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":28,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:49:59.705865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":68398827,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":6876,"num_read_kb":94226,"num_write":603,"num_write_kb":45683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":119880838,"num_keys_recovered":83,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":55}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'627","reported_seq":7254,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.026349+0000","last_change":"2024-09-12T09:32:35.026870+0000","last_active":"2024-09-12T09:32:37.026349+0000","last_peered":"2024-09-12T09:32:37.026349+0000","last_clean":"2024-09-12T09:32:37.026349+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:32:37.026349+0000","last_undegraded":"2024-09-12T09:32:37.026349+0000","last_fullsized":"2024-09-12T09:32:37.026349+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:25.483002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041687759999999999,"stat_sum":{"num_bytes":83471268,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":7396,"num_read_kb":80041,"num_write":722,"num_write_kb":46022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.10","version":"177'647","reported_seq":3908,"reported_epoch":177,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:35.047364+0000","last_change":"2024-09-12T09:32:35.048014+0000","last_active":"2024-09-12T09:32:35.047364+0000","last_peered":"2024-09-12T09:32:35.047364+0000","last_clean":"2024-09-12T09:32:35.047364+0000","last_became_active":"2024-09-12T09:31:59.687172+0000","last_became_peered":"2024-09-12T09:31:59.687172+0000","last_unstale":"2024-09-12T09:32:35.047364+0000","last_undegraded":"2024-09-12T09:32:35.047364+0000","last_fullsized":"2024-09-12T09:32:35.047364+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:12:53.126341+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089693570000000007,"stat_sum":{"num_bytes":34844554,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":37,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":19,"num_read":3688,"num_read_kb":27709,"num_write":804,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[{"shards":"5,6","objects":37}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.11","version":"177'618","reported_seq":3334,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.025298+0000","last_change":"2024-09-12T09:32:35.041663+0000","last_active":"2024-09-12T09:32:37.025298+0000","last_peered":"2024-09-12T09:32:37.025298+0000","last_clean":"2024-09-12T09:32:37.025298+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:32:37.025298+0000","last_undegraded":"2024-09-12T09:32:37.025298+0000","last_fullsized":"2024-09-12T09:32:37.025298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:25.781781+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016959912000000001,"stat_sum":{"num_bytes":49733782,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":5429,"num_read_kb":19835,"num_write":743,"num_write_kb":9430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]},{"pgid":"3.12","version":"171'399","reported_seq":13143,"reported_epoch":179,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:37.025303+0000","last_change":"2024-09-12T09:31:58.683646+0000","last_active":"2024-09-12T09:32:37.025303+0000","last_peered":"2024-09-12T09:32:37.025303+0000","last_clean":"2024-09-12T09:31:55.763534+0000","last_became_active":"2024-09-12T09:31:58.683646+0000","last_became_peered":"2024-09-12T09:31:58.683646+0000","last_unstale":"2024-09-12T09:32:37.025303+0000","last_undegraded":"2024-09-12T09:31:58.681905+0000","last_fullsized":"2024-09-12T09:31:58.681670+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":28,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:43.898256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077262000000000001,"stat_sum":{"num_bytes":91866375,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":27,"num_read":13178,"num_read_kb":129719,"num_write":510,"num_write_kb":65716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":181958090,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7","3"],"object_location_counts":[{"shards":"7","objects":59}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"173'235","reported_seq":1794,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-12T09:32:37.025744+0000","last_change":"2024-09-12T09:32:35.022171+0000","last_active":"2024-09-12T09:32:37.025744+0000","last_peered":"2024-09-12T09:32:37.025744+0000","last_clean":"2024-09-12T09:32:37.025744+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:32:37.025744+0000","last_undegraded":"2024-09-12T09:32:37.025744+0000","last_fullsized":"2024-09-12T09:32:37.025744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:37.488460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099206800000000003,"stat_sum":{"num_bytes":37268147,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":1624,"num_read_kb":9781,"num_write":313,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"1c"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1321665151,"num_objects":1268,"num_object_clones":291,"num_object_copies":2536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":207,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":1268,"num_whiteouts":553,"num_read":248892,"num_read_kb":1577265,"num_write":22004,"num_write_kb":852279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":782986986,"num_keys_recovered":692,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":74559488,"data_stored":1056709348,"data_compressed":10416932,"data_compressed_allocated":66240512,"data_compressed_original":1051053304,"omap_allocated":0,"internal_metadata":0},"log_size":18120,"ondisk_log_size":18120,"up":53,"acting":59,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":1,"acting":1,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":15360,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":15728640,"data_stored":190627491,"data_compressed":1999729,"data_compressed_allocated":12144640,"data_compressed_original":188554998,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56500,"kb_used_data":11420,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94315340,"statfs":{"total":96636764160,"available":96578908160,"internally_reserved":0,"allocated":11694080,"data_stored":125430364,"data_compressed":1228212,"data_compressed_allocated":8355840,"data_compressed_original":123625446,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43732,"kb_used_data":4224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328108,"statfs":{"total":96636764160,"available":96591982592,"internally_reserved":0,"allocated":4325376,"data_stored":20227159,"data_compressed":493499,"data_compressed_allocated":1814528,"data_compressed_original":18968576,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72440,"kb_used_data":14876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94299400,"statfs":{"total":96636764160,"available":96562585600,"internally_reserved":0,"allocated":15233024,"data_stored":205128502,"data_compressed":1155946,"data_compressed_allocated":11730944,"data_compressed_original":203079461,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66788,"kb_used_data":17552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305052,"statfs":{"total":96636764160,"available":96568373248,"internally_reserved":0,"allocated":17973248,"data_stored":230907018,"data_compressed":2194952,"data_compressed_allocated":14540800,"data_compressed_original":229086671,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60036,"kb_used_data":6632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":6791168,"data_stored":70082974,"data_compressed":719195,"data_compressed_allocated":4091904,"data_compressed_original":68606038,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37004,"kb_used_data":5816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334836,"statfs":{"total":96636764160,"available":96598872064,"internally_reserved":0,"allocated":5955584,"data_stored":40624338,"data_compressed":759641,"data_compressed_allocated":3198976,"data_compressed_original":39209046,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46764,"kb_used_data":11420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325076,"statfs":{"total":96636764160,"available":96588877824,"internally_reserved":0,"allocated":11694080,"data_stored":163878562,"data_compressed":1334902,"data_compressed_allocated":9007104,"data_compressed_original":162307072,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11812864,"data_stored":181540902,"data_compressed":1872412,"data_compressed_allocated":10399744,"data_compressed_original":180512892,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3825664,"data_stored":39587649,"data_compressed":759641,"data_compressed_allocated":3198976,"data_compressed_original":39209046,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4665344,"data_stored":69046304,"data_compressed":719195,"data_compressed_allocated":4091904,"data_compressed_original":68606038,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15847424,"data_stored":229870348,"data_compressed":2194952,"data_compressed_allocated":14540800,"data_compressed_original":229086671,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9555968,"data_stored":124387542,"data_compressed":1228212,"data_compressed_allocated":8355840,"data_compressed_original":123625446,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13103104,"data_stored":204091813,"data_compressed":1155946,"data_compressed_allocated":11730944,"data_compressed_original":203079461,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":19190489,"data_compressed":493499,"data_compressed_allocated":1814528,"data_compressed_original":18968576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13549568,"data_stored":188994301,"data_compressed":1993075,"data_compressed_allocated":12107776,"data_compressed_original":187965174,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:32:40.913 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [7, 2] 2024-09-12T09:32:40.913 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '7', '2'] 2024-09-12T09:32:40.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 7 2 2024-09-12T09:32:40.914 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:40.905+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:40.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:41.009 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:41.005+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:41.076 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:41.077 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:41.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:41.109 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:41.105+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:41.115 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:41.115 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:41.155 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:41.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:41.177 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:41.210 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:41.205+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:41.213 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:41.248 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:41.248 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:41.250 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:41.250 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:41.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:41.277 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:41.280 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:41.280 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:41.311 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:41.305+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:41.388 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:41.388 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:41.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:41.448 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:41.445+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:41.513 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:41.511+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:41.532 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:41.532 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:41.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:41.636 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:41.633+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:41.711 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:41.711 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:41.715 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:41.709+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:41.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:41.815 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:41.809+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:41.884 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:41.884 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:41.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:41.940 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:41.937+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:42.016 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:42.015+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:42.062 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.063 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.077 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:42.091 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.12 pg_upmap mapping to [7,2] 2024-09-12T09:32:42.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:32:42.117 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:42.115+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:42.212 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.212 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.217 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:42.215+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:42.227 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:42.319 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:42.315+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:42.344 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.344 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:42.420 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:42.419+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:42.475 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.475 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.491 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:42.521 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:42.519+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:42.612 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.612 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.622 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:42.617+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:42.626 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:42.682 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:32:42.723 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:42.721+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:42.794 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.794 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:42.852 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:42.849+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:42.926 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:42.923+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:42.979 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:42.979 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:42.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:43.027 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:43.027+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:43.117 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:43.117 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:43.128 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:43.125+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:43.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:43.233 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:43.229+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:43.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:43.300 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:43.300 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:43.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:43.360 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:43.357+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:43.430 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:43.427+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:43.444 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:43.457 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:43.457 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:43.461 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:43.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:43.478 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:43.499 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:43.520 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:43.531 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:43.527+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:43.542 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:43.542 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:43.563 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:43.567 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:43.567 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:43.593 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:43.593 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:43.593 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:43.593 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:43.593 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:43.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:43.632 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:43.629+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:43.733 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:43.729+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:43.789 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.58842356400001, 2024-09-12T09:32:43.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:43.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:32:43.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.57345283500001, 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:32:43.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:43.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256605831, 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:32:43.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:32:43.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:43.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25655947199999, 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:32:43.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:32:43.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256452252, 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:32:43.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256358761, 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:32:43.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25629224799999, 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:32:43.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:32:43.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256231439, 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:32:43.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256172147, 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:32:43.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25611163799999, 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:43.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.256033583, 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:32:43.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:32:43.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25594820800001, 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:32:43.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:32:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.255879889, 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:32:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25578730399999, 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:32:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.255731395, 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:32:43.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:32:43.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.255699072, 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25566547899999, 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:32:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:32:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.25561453700001, 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:32:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:32:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.255516382, 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:32:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.255459288, 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:32:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:43.894 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:43.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:43.896 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:43.895+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:43.997 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:43.993+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:44.097 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:44.097 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:44.097 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:44.093+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:44.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:44.198 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:44.195+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:44.244 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:44.244 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:44.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:44.299 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:44.293+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:44.400 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:44.399+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 141.56336520299999, 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 141.21583531499999, 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 141.21338140500001, 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 141.21127151799999, 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.71552487300001, 2024-09-12T09:32:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487750356999996, 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487689794999994, 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487625518000002, 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487557999000003, 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487453372999994, 2024-09-12T09:32:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487389840999995, 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487351900999997, 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487323380999996, 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487273877999996, 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487229489000001, 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487167606, 2024-09-12T09:32:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487120093000001, 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487091823, 2024-09-12T09:32:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487064270000005, 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.487033487000005, 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:44.541 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:44.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:44.543 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:44.539+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:44.644 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:44.641+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:44.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:32:44.718 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:44.718 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:44.718 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:44.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:44.779 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:44.779+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:44.845 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:44.843+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:44.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:44.909 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:44.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:44.946 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:44.941+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:45.047 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:45.045+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.65880649899999, 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:45.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.658781268, 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:45.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.658743348, 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:45.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.658713347, 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.650678926, 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.65035805300001, 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.649882748, 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.649854, 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.649303534, 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.649260712, 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.649214821, 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.648703263, 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.648655484, 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.648599838, 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.648567817, 2024-09-12T09:32:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.648003668, 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.64760976700001, 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.647218069, 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.279351950999995, 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.420875154999997, 2024-09-12T09:32:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:45.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:45.201 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:45.199+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:45.300 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:45.297+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:45.333 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:45.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:45.401 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:45.399+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:45.502 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:45.497+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:45.516 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:45.516 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:45.516 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:45.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:45.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:45.620 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:45.617+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 161.40537023499999, 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:45.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.945874006, 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:45.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.572409808, 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:45.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.57211673200001, 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:45.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:45.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.5715548, 2024-09-12T09:32:45.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:45.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:45.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.57150343699999, 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:45.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.570930557, 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:45.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:45.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.57077825799999, 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:45.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.570421168, 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:45.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:45.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56979619800001, 2024-09-12T09:32:45.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:45.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:45.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56974486199999, 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:45.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:45.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:45.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56895007200001, 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:45.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:45.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.568875182, 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56882286199999, 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:45.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56877426400001, 2024-09-12T09:32:45.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:45.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.568519799, 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:45.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 152.56506550899999, 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:45.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 142.507496962, 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:45.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:45.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.03622572500001, 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:45.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:202431 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.640938458000001, 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.971370387, 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:45.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 202431 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:45.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053485+0000", 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.716e-06 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053491+0000", 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.733e-06 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:45.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053504+0000", 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3002e-05 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053510+0000", 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0039999999999996e-06 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053627+0000", 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117124 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053640+0000", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2656999999999999e-05 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024816+0000", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97117616900000003 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024854+0000", 2024-09-12T09:32:45.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7982000000000003e-05 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:45.796 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:45.797 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:45.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:45.799 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:45.793+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:45.799 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:45.799 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:45.799 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:45.800 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:45.800 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:45.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:45.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:45.825 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:45.826 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:45.846 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:45.866 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:45.870 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:45.870 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:45.911 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:45.911+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:45.972 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:45.973 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:45.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:46.036 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:46.033+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:46.100 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:46.097+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:46.111 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.112 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:46.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:46.201 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:46.199+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.289 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:46.302 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:46.297+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:46.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:46.403 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:46.399+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:46.467 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.467 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:46.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:46.528 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:46.525+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:46.605 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.605 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:46.605 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:46.601+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:46.618 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:46.706 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:46.703+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:46.800 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.801 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:46.807 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:46.801+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:46.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:46.909 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:46.907+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:46.991 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:46.991 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:47.009 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:47.005+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:47.110 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:47.105+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:47.120 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:47.120 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:47.121 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:32:47.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:32:47.133 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:47.133 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:47.211 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:47.207+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:47.311 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:47.309+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:47.317 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:47.317 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:47.412 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:47.411+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:47.454 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 189, 2024-09-12T09:32:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:32:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:32:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:32:46.707599+0000", 2024-09-12T09:32:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:31:57.668482+0000", 2024-09-12T09:32:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:51.831699+0000", 2024-09-12T09:32:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:32:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5801984, 2024-09-12T09:32:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:32:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-12T09:32:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:32:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:32:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:32:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:32:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:32:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:32:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:32:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:32:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:32:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:32:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:32:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:32:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:32:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:32:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:32:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:32:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:32:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:32:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:32:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:32:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:32:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:32:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:32:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:32:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:32:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:32:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:32:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:32:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:47.474 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:32:47.474 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:32:47.474 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:32:47.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.474 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:47.475 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:32:47.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.475 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:47.475 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:47.475 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 7.690000057220459, 2024-09-12T09:32:47.476 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 7.690000057220459, 2024-09-12T09:32:47.476 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-12T09:32:47.476 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-12T09:32:47.476 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-12T09:32:47.476 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:47.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:47.479 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:47.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:47.479 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:47.479 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:47.479 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:47.480 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:47.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:32:47.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:32:47.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:32:47.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:32:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:32:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:32:47.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:47.484 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:32:47.485 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:32:47.485 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:47.485 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:47.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:47.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:47.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:47.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:47.486 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:47.486 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:47.487 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:47.488 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:47.488 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:47.488 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:47.488 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:47.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:47.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:47.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:47.490 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:47.490 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:47.490 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:47.490 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:47.491 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:47.491 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:47.491 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:47.491 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:47.491 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:32:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:32:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:32:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:32:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:32:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:32:47.495 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:32:47.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:47.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:47.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:47.495 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:47.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:47.496 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:47.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:47.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:47.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:47.497 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:47.497 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:47.497 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:47.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:47.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:47.498 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:47.499 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:47.499 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:47.499 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:47.499 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:47.500 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:47.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "189", 2024-09-12T09:32:47.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:47.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:47.501 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:47.501 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:47.501 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:47.501 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 55, 2024-09-12T09:32:47.501 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 189, 2024-09-12T09:32:47.502 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:47.502 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:47.502 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:47.502 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:47.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:47.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:47.503 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:47.503 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:47.503 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:47.503 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:47.504 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:47.504 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:47.504 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:47.504 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:47.504 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:47.505 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:47.505 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:47.505 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:47.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:47.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.506 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:47.506 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:47.506 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:47.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:47.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:47.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:47.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:47.507 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:47.507 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:47.507 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:47.508 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:47.508 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:47.508 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:47.508 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:47.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2400000095367432, 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2400000095367432, 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:32:47.509 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-12T09:32:47.510 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-12T09:32:47.510 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:47.510 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:47.510 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:47.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.511 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.511 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.511 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:47.512 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:32:47.512 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-12T09:32:47.512 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.512 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.512 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.513 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.513 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.513 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.513 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 140, 2024-09-12T09:32:47.513 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 141, 2024-09-12T09:32:47.514 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.514 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.514 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.515 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:47.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:32:47.519 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 179, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:32:47.524 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.525 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 138, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:32:47.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:32:47.530 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.531 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:32:47.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:32:47.536 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.537 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 180, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:32:47.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:32:47.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:32:47.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:32:47.542 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.543 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 155, 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.548 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.0386199951171875, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 154, 2024-09-12T09:32:47.549 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:32:47.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 187, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:47.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.556 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 141 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.562 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.564 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.565 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.12", 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.568 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1", 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.3", 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:47.569 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.10", 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.570 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.17", 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:32:47.572 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:47.573 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 21, 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 35 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 21, 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-12T09:32:47.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:47.576 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:47.576 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.578 INFO:tasks.thrashosds.thrasher:j is {'epoch': 189, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:32:46.707599+0000', 'last_up_change': '2024-09-12T09:31:57.668482+0000', 'last_in_change': '2024-09-12T09:31:51.831699+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 189, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 179, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 180, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0386199951171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 187, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.e', 'osds': [5, 0]}, {'pgid': '3.12', 'osds': [7, 2]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1', 'osds': [4, 3]}, {'pgid': '3.3', 'osds': [4, 6]}, {'pgid': '3.b', 'osds': [3, 4]}, {'pgid': '3.10', 'osds': [6, 5]}, {'pgid': '3.17', 'osds': [5, 3]}, {'pgid': '3.18', 'osds': [3, 1]}, {'pgid': '3.1b', 'osds': [4, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 35}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:32:47.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:32:47.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:47.579 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:47.577+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:47.677 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:47.673+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:47.769 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.778 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:47.775+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:47.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:47.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:32:47.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:47.919 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:47.919+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:32:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:47.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-12T09:32:47.729543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1080194397,"num_objects":1037,"num_object_clones":220,"num_object_copies":2074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":1037,"num_whiteouts":499,"num_read":251600,"num_read_kb":1579570,"num_write":22484,"num_write_kb":854008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4356,"num_bytes_recovered":835969734,"num_keys_recovered":715,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18745,"ondisk_log_size":18745,"up":69,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":451908,"kb_used_data":82984,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754522812,"statfs":{"total":773094113280,"available":772631359488,"internally_reserved":0,"allocated":84975616,"data_stored":1007824854,"data_compressed":7442581,"data_compressed_allocated":60788736,"data_compressed_original":994519821,"omap_allocated":12752,"internal_metadata":377540144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-86742849,"num_objects":-148,"num_object_clones":-48,"num_object_copies":-296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":-19,"num_objects_unfound":0,"num_objects_dirty":-148,"num_whiteouts":-48,"num_read":1723,"num_read_kb":1468,"num_write":206,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":46459029,"num_keys_recovered":11,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.042696"},"pg_stats":[{"pgid":"3.1f","version":"183'743","reported_seq":6382,"reported_epoch":188,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:45.702460+0000","last_change":"2024-09-12T09:31:58.689191+0000","last_active":"2024-09-12T09:32:45.702460+0000","last_peered":"2024-09-12T09:32:45.702460+0000","last_clean":"2024-09-12T09:31:51.668701+0000","last_became_active":"2024-09-12T09:31:58.689191+0000","last_became_peered":"2024-09-12T09:31:58.689191+0000","last_unstale":"2024-09-12T09:32:45.702460+0000","last_undegraded":"2024-09-12T09:31:58.686707+0000","last_fullsized":"2024-09-12T09:31:58.686488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:13.644039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":80380605,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":20,"num_read":8213,"num_read_kb":95267,"num_write":947,"num_write_kb":47653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":37755293,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":45}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"188'716","reported_seq":6150,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.721375+0000","last_change":"2024-09-12T09:32:45.722211+0000","last_active":"2024-09-12T09:32:45.721375+0000","last_peered":"2024-09-12T09:32:45.721375+0000","last_clean":"2024-09-12T09:32:45.721375+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:32:45.721375+0000","last_undegraded":"2024-09-12T09:32:45.721375+0000","last_fullsized":"2024-09-12T09:32:45.721375+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:18:35.687127+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.020812536,"stat_sum":{"num_bytes":36142742,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":8673,"num_read_kb":41263,"num_write":824,"num_write_kb":38763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.1d","version":"188'510","reported_seq":8092,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.833611+0000","last_change":"2024-09-12T09:32:45.734175+0000","last_active":"2024-09-12T09:32:45.833611+0000","last_peered":"2024-09-12T09:32:45.833611+0000","last_clean":"2024-09-12T09:32:45.833611+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:32:45.833611+0000","last_undegraded":"2024-09-12T09:32:45.833611+0000","last_fullsized":"2024-09-12T09:32:45.833611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:42.570601+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.033432034999999999,"stat_sum":{"num_bytes":42981197,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":9121,"num_read_kb":78651,"num_write":486,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.1c","version":"188'697","reported_seq":6903,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.906115+0000","last_change":"2024-09-12T09:32:45.718395+0000","last_active":"2024-09-12T09:32:45.906115+0000","last_peered":"2024-09-12T09:32:45.906115+0000","last_clean":"2024-09-12T09:32:45.906115+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:32:45.906115+0000","last_undegraded":"2024-09-12T09:32:45.906115+0000","last_fullsized":"2024-09-12T09:32:45.906115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:52:23.130617+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.018109363999999999,"stat_sum":{"num_bytes":19732605,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":8231,"num_read_kb":52429,"num_write":761,"num_write_kb":16449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.1b","version":"189'341","reported_seq":6368,"reported_epoch":189,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:47.059723+0000","last_change":"2024-09-12T09:32:46.733215+0000","last_active":"2024-09-12T09:32:47.059723+0000","last_peered":"2024-09-12T09:32:47.059723+0000","last_clean":"2024-09-12T09:32:47.059723+0000","last_became_active":"2024-09-12T09:31:59.687414+0000","last_became_peered":"2024-09-12T09:31:59.687414+0000","last_unstale":"2024-09-12T09:32:47.059723+0000","last_undegraded":"2024-09-12T09:32:47.059723+0000","last_fullsized":"2024-09-12T09:32:47.059723+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:47:37.973380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017887980000000001,"stat_sum":{"num_bytes":15084005,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":28,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":6103,"num_read_kb":20719,"num_write":334,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[{"shards":"4,7","objects":28}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.1a","version":"189'579","reported_seq":9380,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:47.029918+0000","last_change":"2024-09-12T09:32:46.732300+0000","last_active":"2024-09-12T09:32:47.029918+0000","last_peered":"2024-09-12T09:32:47.029918+0000","last_clean":"2024-09-12T09:32:47.029918+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:32:47.029918+0000","last_undegraded":"2024-09-12T09:32:47.029918+0000","last_fullsized":"2024-09-12T09:32:47.029918+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:27.518507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010387459999999999,"stat_sum":{"num_bytes":8311250,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":8991,"num_read_kb":32032,"num_write":615,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.19","version":"188'407","reported_seq":3784,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.794626+0000","last_change":"2024-09-12T09:32:45.705838+0000","last_active":"2024-09-12T09:32:45.794626+0000","last_peered":"2024-09-12T09:32:45.794626+0000","last_clean":"2024-09-12T09:32:45.794626+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:32:45.794626+0000","last_undegraded":"2024-09-12T09:32:45.794626+0000","last_fullsized":"2024-09-12T09:32:45.794626+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:15.275641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016770789999999999,"stat_sum":{"num_bytes":13597632,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":4039,"num_read_kb":15039,"num_write":459,"num_write_kb":6310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.18","version":"188'462","reported_seq":4098,"reported_epoch":188,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:45.710641+0000","last_change":"2024-09-12T09:32:45.711464+0000","last_active":"2024-09-12T09:32:45.710641+0000","last_peered":"2024-09-12T09:32:45.710641+0000","last_clean":"2024-09-12T09:32:45.710641+0000","last_became_active":"2024-09-12T09:31:59.685845+0000","last_became_peered":"2024-09-12T09:31:59.685845+0000","last_unstale":"2024-09-12T09:32:45.710641+0000","last_undegraded":"2024-09-12T09:32:45.710641+0000","last_fullsized":"2024-09-12T09:32:45.710641+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:19.758183+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.010529636,"stat_sum":{"num_bytes":8313761,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":26,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":2635,"num_read_kb":10192,"num_write":489,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":26}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.17","version":"187'610","reported_seq":4950,"reported_epoch":188,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:45.720140+0000","last_change":"2024-09-12T09:32:45.720861+0000","last_active":"2024-09-12T09:32:45.720140+0000","last_peered":"2024-09-12T09:32:45.720140+0000","last_clean":"2024-09-12T09:32:45.720140+0000","last_became_active":"2024-09-12T09:31:59.686284+0000","last_became_peered":"2024-09-12T09:31:59.686284+0000","last_unstale":"2024-09-12T09:32:45.720140+0000","last_undegraded":"2024-09-12T09:32:45.720140+0000","last_fullsized":"2024-09-12T09:32:45.720140+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:37.150342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019602142999999999,"stat_sum":{"num_bytes":34289362,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":6896,"num_read_kb":53657,"num_write":712,"num_write_kb":39724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":20}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.16","version":"188'521","reported_seq":15784,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.745225+0000","last_change":"2024-09-12T09:32:45.745667+0000","last_active":"2024-09-12T09:32:45.745225+0000","last_peered":"2024-09-12T09:32:45.745225+0000","last_clean":"2024-09-12T09:32:45.745225+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:32:45.745225+0000","last_undegraded":"2024-09-12T09:32:45.745225+0000","last_fullsized":"2024-09-12T09:32:45.745225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:10.030046+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.044020445999999998,"stat_sum":{"num_bytes":44059788,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":15916,"num_read_kb":96835,"num_write":564,"num_write_kb":40304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.15","version":"188'790","reported_seq":8670,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.729766+0000","last_change":"2024-09-12T09:32:46.730349+0000","last_active":"2024-09-12T09:32:46.729766+0000","last_peered":"2024-09-12T09:32:46.729766+0000","last_clean":"2024-09-12T09:32:46.729766+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:32:46.729766+0000","last_undegraded":"2024-09-12T09:32:46.729766+0000","last_fullsized":"2024-09-12T09:32:46.729766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:12.742074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024602690000000002,"stat_sum":{"num_bytes":66512118,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":9168,"num_read_kb":102695,"num_write":1015,"num_write_kb":47722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.14","version":"188'799","reported_seq":12229,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.733313+0000","last_change":"2024-09-12T09:32:46.734186+0000","last_active":"2024-09-12T09:32:46.733313+0000","last_peered":"2024-09-12T09:32:46.733313+0000","last_clean":"2024-09-12T09:32:46.733313+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:32:46.733313+0000","last_undegraded":"2024-09-12T09:32:46.733313+0000","last_fullsized":"2024-09-12T09:32:46.733313+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:45.192897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025059930000000002,"stat_sum":{"num_bytes":21175339,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":13257,"num_read_kb":53912,"num_write":1116,"num_write_kb":31334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"2.7","version":"0'0","reported_seq":150,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.727867+0000","last_change":"2024-09-12T09:32:22.911749+0000","last_active":"2024-09-12T09:32:46.727867+0000","last_peered":"2024-09-12T09:32:46.727867+0000","last_clean":"2024-09-12T09:32:46.727867+0000","last_became_active":"2024-09-12T09:32:22.911415+0000","last_became_peered":"2024-09-12T09:32:22.911415+0000","last_unstale":"2024-09-12T09:32:46.727867+0000","last_undegraded":"2024-09-12T09:32:46.727867+0000","last_fullsized":"2024-09-12T09:32:46.727867+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:37.113174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"188'362","reported_seq":6975,"reported_epoch":188,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:45.705211+0000","last_change":"2024-09-12T09:31:58.680982+0000","last_active":"2024-09-12T09:32:45.705211+0000","last_peered":"2024-09-12T09:32:45.705211+0000","last_clean":"2024-09-12T09:31:51.383028+0000","last_became_active":"2024-09-12T09:31:58.680982+0000","last_became_peered":"2024-09-12T09:31:58.680982+0000","last_unstale":"2024-09-12T09:32:45.705211+0000","last_undegraded":"2024-09-12T09:31:58.679925+0000","last_fullsized":"2024-09-12T09:31:58.679826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:33.674896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":55220507,"num_objects":42,"num_object_clones":15,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":19,"num_read":6905,"num_read_kb":29670,"num_write":309,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":42}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.704088+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:32:45.704088+0000","last_peered":"2024-09-12T09:32:45.704088+0000","last_clean":"2024-09-12T09:32:45.704088+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:32:45.704088+0000","last_undegraded":"2024-09-12T09:32:45.704088+0000","last_fullsized":"2024-09-12T09:32:45.704088+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"182'402","reported_seq":4011,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.730376+0000","last_change":"2024-09-12T09:32:46.730376+0000","last_active":"2024-09-12T09:32:46.730376+0000","last_peered":"2024-09-12T09:32:46.730376+0000","last_clean":"2024-09-12T09:32:46.730376+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:32:46.730376+0000","last_undegraded":"2024-09-12T09:32:46.730376+0000","last_fullsized":"2024-09-12T09:32:46.730376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:23.893612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027293859999999999,"stat_sum":{"num_bytes":24555530,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":3901,"num_read_kb":30598,"num_write":445,"num_write_kb":26827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"2.5","version":"0'0","reported_seq":265,"reported_epoch":189,"state":"active+undersized","last_fresh":"2024-09-12T09:32:46.727587+0000","last_change":"2024-09-12T09:31:58.689633+0000","last_active":"2024-09-12T09:32:46.727587+0000","last_peered":"2024-09-12T09:32:46.727587+0000","last_clean":"2024-09-12T09:31:56.850902+0000","last_became_active":"2024-09-12T09:31:58.689633+0000","last_became_peered":"2024-09-12T09:31:58.689633+0000","last_unstale":"2024-09-12T09:32:46.727587+0000","last_undegraded":"2024-09-12T09:32:46.727587+0000","last_fullsized":"2024-09-12T09:31:58.684053+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:23.955347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"188'1124","reported_seq":8015,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.709501+0000","last_change":"2024-09-12T09:32:45.710104+0000","last_active":"2024-09-12T09:32:45.709501+0000","last_peered":"2024-09-12T09:32:45.709501+0000","last_clean":"2024-09-12T09:32:45.709501+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:32:45.709501+0000","last_undegraded":"2024-09-12T09:32:45.709501+0000","last_fullsized":"2024-09-12T09:32:45.709501+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:47.273074+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0054926899999999997,"stat_sum":{"num_bytes":13039090,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":11145,"num_read_kb":45471,"num_write":1739,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":188,"state":"active+undersized","last_fresh":"2024-09-12T09:32:45.704280+0000","last_change":"2024-09-12T09:31:58.680941+0000","last_active":"2024-09-12T09:32:45.704280+0000","last_peered":"2024-09-12T09:32:45.704280+0000","last_clean":"2024-09-12T09:31:56.702759+0000","last_became_active":"2024-09-12T09:31:58.680941+0000","last_became_peered":"2024-09-12T09:31:58.680941+0000","last_unstale":"2024-09-12T09:32:45.704280+0000","last_undegraded":"2024-09-12T09:32:45.704280+0000","last_fullsized":"2024-09-12T09:31:58.679822+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:32.828980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"188'258","reported_seq":14038,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.726305+0000","last_change":"2024-09-12T09:32:45.726598+0000","last_active":"2024-09-12T09:32:45.726305+0000","last_peered":"2024-09-12T09:32:45.726305+0000","last_clean":"2024-09-12T09:32:45.726305+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:32:45.726305+0000","last_undegraded":"2024-09-12T09:32:45.726305+0000","last_fullsized":"2024-09-12T09:32:45.726305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:35.834284+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.025025287,"stat_sum":{"num_bytes":28066020,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":13557,"num_read_kb":60534,"num_write":214,"num_write_kb":14069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.3","version":"189'1075","reported_seq":6192,"reported_epoch":189,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:46.911852+0000","last_change":"2024-09-12T09:32:46.734771+0000","last_active":"2024-09-12T09:32:46.911852+0000","last_peered":"2024-09-12T09:32:46.911852+0000","last_clean":"2024-09-12T09:32:46.911852+0000","last_became_active":"2024-09-12T09:31:59.688563+0000","last_became_peered":"2024-09-12T09:31:59.688563+0000","last_unstale":"2024-09-12T09:32:46.911852+0000","last_undegraded":"2024-09-12T09:32:46.911852+0000","last_fullsized":"2024-09-12T09:32:46.911852+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:44.652187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028294639999999999,"stat_sum":{"num_bytes":21190526,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":7819,"num_read_kb":25105,"num_write":1445,"num_write_kb":11298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":39}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"2.2","version":"21'2","reported_seq":396,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.699967+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:32:45.699967+0000","last_peered":"2024-09-12T09:32:45.699967+0000","last_clean":"2024-09-12T09:32:45.699967+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:32:45.699967+0000","last_undegraded":"2024-09-12T09:32:45.699967+0000","last_fullsized":"2024-09-12T09:32:45.699967+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"188'388","reported_seq":5047,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.714318+0000","last_change":"2024-09-12T09:32:45.714668+0000","last_active":"2024-09-12T09:32:45.714318+0000","last_peered":"2024-09-12T09:32:45.714318+0000","last_clean":"2024-09-12T09:32:45.714318+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:32:45.714318+0000","last_undegraded":"2024-09-12T09:32:45.714318+0000","last_fullsized":"2024-09-12T09:32:45.714318+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:47.481854+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.010024926999999999,"stat_sum":{"num_bytes":28986728,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":4684,"num_read_kb":38015,"num_write":333,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"2.1","version":"0'0","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.697683+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:32:45.697683+0000","last_peered":"2024-09-12T09:32:45.697683+0000","last_clean":"2024-09-12T09:32:45.697683+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:32:45.697683+0000","last_undegraded":"2024-09-12T09:32:45.697683+0000","last_fullsized":"2024-09-12T09:32:45.697683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"179'488","reported_seq":13233,"reported_epoch":189,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:47.016243+0000","last_change":"2024-09-12T09:32:46.733218+0000","last_active":"2024-09-12T09:32:47.016243+0000","last_peered":"2024-09-12T09:32:47.016243+0000","last_clean":"2024-09-12T09:32:47.016243+0000","last_became_active":"2024-09-12T09:31:59.684097+0000","last_became_peered":"2024-09-12T09:31:59.684097+0000","last_unstale":"2024-09-12T09:32:47.016243+0000","last_undegraded":"2024-09-12T09:32:47.016243+0000","last_fullsized":"2024-09-12T09:32:47.016243+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:32.085976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001941538,"stat_sum":{"num_bytes":36625257,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":13928,"num_read_kb":39079,"num_write":516,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":10010383,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":27}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.727238+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:32:46.727238+0000","last_peered":"2024-09-12T09:32:46.727238+0000","last_clean":"2024-09-12T09:32:46.727238+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:32:46.727238+0000","last_undegraded":"2024-09-12T09:32:46.727238+0000","last_fullsized":"2024-09-12T09:32:46.727238+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"187'375","reported_seq":5691,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.893340+0000","last_change":"2024-09-12T09:32:45.719133+0000","last_active":"2024-09-12T09:32:45.893340+0000","last_peered":"2024-09-12T09:32:45.893340+0000","last_clean":"2024-09-12T09:32:45.893340+0000","last_became_active":"2024-09-12T09:32:13.696998+0000","last_became_peered":"2024-09-12T09:32:13.696998+0000","last_unstale":"2024-09-12T09:32:45.893340+0000","last_undegraded":"2024-09-12T09:32:45.893340+0000","last_fullsized":"2024-09-12T09:32:45.893340+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:46:37.264530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018648121,"stat_sum":{"num_bytes":29564344,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":6208,"num_read_kb":46932,"num_write":368,"num_write_kb":14935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":9357778,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"2.3","version":"19'1","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.699946+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:32:45.699946+0000","last_peered":"2024-09-12T09:32:45.699946+0000","last_clean":"2024-09-12T09:32:45.699946+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:32:45.699946+0000","last_undegraded":"2024-09-12T09:32:45.699946+0000","last_fullsized":"2024-09-12T09:32:45.699946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":295,"reported_epoch":189,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:46.727247+0000","last_change":"2024-09-12T09:31:58.683002+0000","last_active":"2024-09-12T09:32:46.727247+0000","last_peered":"2024-09-12T09:32:46.727247+0000","last_clean":"2024-09-12T09:31:56.850615+0000","last_became_active":"2024-09-12T09:31:58.683002+0000","last_became_peered":"2024-09-12T09:31:58.683002+0000","last_unstale":"2024-09-12T09:32:46.727247+0000","last_undegraded":"2024-09-12T09:31:58.681286+0000","last_fullsized":"2024-09-12T09:31:58.681061+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:41.462657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"188'1254","reported_seq":6179,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.883801+0000","last_change":"2024-09-12T09:32:45.711332+0000","last_active":"2024-09-12T09:32:45.883801+0000","last_peered":"2024-09-12T09:32:45.883801+0000","last_clean":"2024-09-12T09:32:45.883801+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:32:45.883801+0000","last_undegraded":"2024-09-12T09:32:45.883801+0000","last_fullsized":"2024-09-12T09:32:45.883801+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:14:20.855125+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010453375000000001,"stat_sum":{"num_bytes":37915960,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":11013,"num_read_kb":56852,"num_write":1901,"num_write_kb":24766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.9","version":"188'714","reported_seq":4973,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.733375+0000","last_change":"2024-09-12T09:32:46.734058+0000","last_active":"2024-09-12T09:32:46.733375+0000","last_peered":"2024-09-12T09:32:46.733375+0000","last_clean":"2024-09-12T09:32:46.733375+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:32:46.733375+0000","last_undegraded":"2024-09-12T09:32:46.733375+0000","last_fullsized":"2024-09-12T09:32:46.733375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:02.954620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023448779999999999,"stat_sum":{"num_bytes":12034925,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":6054,"num_read_kb":30020,"num_write":912,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.a","version":"189'453","reported_seq":5734,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.892718+0000","last_change":"2024-09-12T09:32:46.732218+0000","last_active":"2024-09-12T09:32:46.892718+0000","last_peered":"2024-09-12T09:32:46.892718+0000","last_clean":"2024-09-12T09:32:46.892718+0000","last_became_active":"2024-09-12T09:32:22.909677+0000","last_became_peered":"2024-09-12T09:32:22.909677+0000","last_unstale":"2024-09-12T09:32:46.892718+0000","last_undegraded":"2024-09-12T09:32:46.892718+0000","last_fullsized":"2024-09-12T09:32:46.892718+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:00:51.805071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094264100000000003,"stat_sum":{"num_bytes":18175791,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":5035,"num_read_kb":17276,"num_write":521,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.b","version":"187'467","reported_seq":6589,"reported_epoch":188,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:45.711483+0000","last_change":"2024-09-12T09:32:45.712109+0000","last_active":"2024-09-12T09:32:45.711483+0000","last_peered":"2024-09-12T09:32:45.711483+0000","last_clean":"2024-09-12T09:32:45.711483+0000","last_became_active":"2024-09-12T09:31:59.685428+0000","last_became_peered":"2024-09-12T09:31:59.685428+0000","last_unstale":"2024-09-12T09:32:45.711483+0000","last_undegraded":"2024-09-12T09:32:45.711483+0000","last_fullsized":"2024-09-12T09:32:45.711483+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:17:08.057839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010762275999999999,"stat_sum":{"num_bytes":62848525,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":41,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":6709,"num_read_kb":72316,"num_write":620,"num_write_kb":32938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":41}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.c","version":"188'444","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-12T09:32:45.903623+0000","last_change":"2024-09-12T09:32:45.719048+0000","last_active":"2024-09-12T09:32:45.903623+0000","last_peered":"2024-09-12T09:32:45.903623+0000","last_clean":"2024-09-12T09:32:45.903623+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:32:45.903623+0000","last_undegraded":"2024-09-12T09:32:45.903623+0000","last_fullsized":"2024-09-12T09:32:45.903623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:12:09.449643+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.018317521,"stat_sum":{"num_bytes":21836085,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":5235,"num_read_kb":44415,"num_write":421,"num_write_kb":19905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.d","version":"188'547","reported_seq":8915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.729502+0000","last_change":"2024-09-12T09:32:46.729502+0000","last_active":"2024-09-12T09:32:46.729502+0000","last_peered":"2024-09-12T09:32:46.729502+0000","last_clean":"2024-09-12T09:32:46.729502+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:32:46.729502+0000","last_undegraded":"2024-09-12T09:32:46.729502+0000","last_fullsized":"2024-09-12T09:32:46.729502+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:49.155290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001913664,"stat_sum":{"num_bytes":21692814,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":9511,"num_read_kb":28897,"num_write":577,"num_write_kb":7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.e","version":"187'531","reported_seq":5855,"reported_epoch":188,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:45.865301+0000","last_change":"2024-09-12T09:31:58.688892+0000","last_active":"2024-09-12T09:32:45.865301+0000","last_peered":"2024-09-12T09:32:45.865301+0000","last_clean":"2024-09-12T09:31:56.715994+0000","last_became_active":"2024-09-12T09:31:58.688892+0000","last_became_peered":"2024-09-12T09:31:58.688892+0000","last_unstale":"2024-09-12T09:32:45.865301+0000","last_undegraded":"2024-09-12T09:31:58.686246+0000","last_fullsized":"2024-09-12T09:31:58.686019+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:54:03.257717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":67700443,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":6944,"num_read_kb":94284,"num_write":606,"num_write_kb":45683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":119880838,"num_keys_recovered":83,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":52}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"189'641","reported_seq":7323,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.735461+0000","last_change":"2024-09-12T09:32:46.735927+0000","last_active":"2024-09-12T09:32:46.735461+0000","last_peered":"2024-09-12T09:32:46.735461+0000","last_clean":"2024-09-12T09:32:46.735461+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:32:46.735461+0000","last_undegraded":"2024-09-12T09:32:46.735461+0000","last_fullsized":"2024-09-12T09:32:46.735461+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:40.663174+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0078791759999999999,"stat_sum":{"num_bytes":50619634,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":15,"num_read":7396,"num_read_kb":80041,"num_write":728,"num_write_kb":46022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.10","version":"184'657","reported_seq":3971,"reported_epoch":188,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:45.718043+0000","last_change":"2024-09-12T09:32:45.719726+0000","last_active":"2024-09-12T09:32:45.718043+0000","last_peered":"2024-09-12T09:32:45.718043+0000","last_clean":"2024-09-12T09:32:45.718043+0000","last_became_active":"2024-09-12T09:31:59.687172+0000","last_became_peered":"2024-09-12T09:31:59.687172+0000","last_unstale":"2024-09-12T09:32:45.718043+0000","last_undegraded":"2024-09-12T09:32:45.718043+0000","last_fullsized":"2024-09-12T09:32:45.718043+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:03:35.151165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031714709999999999,"stat_sum":{"num_bytes":26015975,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":19,"num_read":3691,"num_read_kb":27710,"num_write":810,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[{"shards":"5,6","objects":29}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"22"}]},{"pgid":"3.11","version":"189'652","reported_seq":3508,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.904167+0000","last_change":"2024-09-12T09:32:46.729665+0000","last_active":"2024-09-12T09:32:46.904167+0000","last_peered":"2024-09-12T09:32:46.904167+0000","last_clean":"2024-09-12T09:32:46.904167+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:32:46.904167+0000","last_undegraded":"2024-09-12T09:32:46.904167+0000","last_fullsized":"2024-09-12T09:32:46.904167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:51.729753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023509989999999999,"stat_sum":{"num_bytes":29419046,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":14,"num_read":5641,"num_read_kb":20015,"num_write":766,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.12","version":"188'416","reported_seq":13590,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:47.030852+0000","last_change":"2024-09-12T09:32:46.728169+0000","last_active":"2024-09-12T09:32:47.030852+0000","last_peered":"2024-09-12T09:32:47.030852+0000","last_clean":"2024-09-12T09:32:47.030852+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:32:47.030852+0000","last_undegraded":"2024-09-12T09:32:47.030852+0000","last_fullsized":"2024-09-12T09:32:47.030852+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:53:51.633136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085353499999999999,"stat_sum":{"num_bytes":70366143,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":20,"num_read":13291,"num_read_kb":129813,"num_write":513,"num_write_kb":65716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"3.13","version":"188'247","reported_seq":1859,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-12T09:32:46.728243+0000","last_change":"2024-09-12T09:32:46.728758+0000","last_active":"2024-09-12T09:32:46.728243+0000","last_peered":"2024-09-12T09:32:46.728243+0000","last_clean":"2024-09-12T09:32:46.728243+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:32:46.728243+0000","last_undegraded":"2024-09-12T09:32:46.728243+0000","last_fullsized":"2024-09-12T09:32:46.728243+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:14.779421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001235185,"stat_sum":{"num_bytes":33150263,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1624,"num_read_kb":9781,"num_write":315,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"15","length":"23"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1079604010,"num_objects":1033,"num_object_clones":220,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":210,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":499,"num_read":251534,"num_read_kb":1579515,"num_write":22381,"num_write_kb":852378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4340,"num_bytes_recovered":832427450,"num_keys_recovered":715,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64524288,"data_stored":947191519,"data_compressed":7077439,"data_compressed_allocated":57823232,"data_compressed_original":942410633,"omap_allocated":0,"internal_metadata":0},"log_size":18670,"ondisk_log_size":18670,"up":54,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":1,"acting":1,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70888,"kb_used_data":13328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300952,"statfs":{"total":96636764160,"available":96564174848,"internally_reserved":0,"allocated":13647872,"data_stored":169532733,"data_compressed":1175117,"data_compressed_allocated":10293248,"data_compressed_original":167575552,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55828,"kb_used_data":10716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94316012,"statfs":{"total":96636764160,"available":96579596288,"internally_reserved":0,"allocated":10973184,"data_stored":118903063,"data_compressed":876537,"data_compressed_allocated":7688192,"data_compressed_original":117125094,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43060,"kb_used_data":3540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328780,"statfs":{"total":96636764160,"available":96592670720,"internally_reserved":0,"allocated":3624960,"data_stored":10185016,"data_compressed":312993,"data_compressed_allocated":1097728,"data_compressed_original":8921088,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71068,"kb_used_data":13512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94300772,"statfs":{"total":96636764160,"available":96563990528,"internally_reserved":0,"allocated":13836288,"data_stored":181027400,"data_compressed":833267,"data_compressed_allocated":10354688,"data_compressed_original":178998087,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65224,"kb_used_data":15984,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94306616,"statfs":{"total":96636764160,"available":96569974784,"internally_reserved":0,"allocated":16367616,"data_stored":208542786,"data_compressed":1686625,"data_compressed_allocated":13017088,"data_compressed_original":206758708,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":9188,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":9408512,"data_stored":121394321,"data_compressed":732107,"data_compressed_allocated":6639616,"data_compressed_original":119871574,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36488,"kb_used_data":5296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335352,"statfs":{"total":96636764160,"available":96599400448,"internally_reserved":0,"allocated":5423104,"data_stored":34360973,"data_compressed":491033,"data_compressed_allocated":2691072,"data_compressed_original":32962646,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46764,"kb_used_data":11420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325076,"statfs":{"total":96636764160,"available":96588877824,"internally_reserved":0,"allocated":11694080,"data_stored":163878562,"data_compressed":1334902,"data_compressed_allocated":9007104,"data_compressed_original":162307072,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11812864,"data_stored":181540902,"data_compressed":1872412,"data_compressed_allocated":10399744,"data_compressed_original":180512892,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3072000,"data_stored":32442177,"data_compressed":473413,"data_compressed_allocated":2625536,"data_compressed_original":32168022,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6688768,"data_stored":117212265,"data_compressed":538492,"data_compressed_allocated":6266880,"data_compressed_original":116852822,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10149888,"data_stored":142761649,"data_compressed":1040411,"data_compressed_allocated":9256960,"data_compressed_original":142190388,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8695808,"data_stored":117790012,"data_compressed":876537,"data_compressed_allocated":7688192,"data_compressed_original":117125094,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11481088,"data_stored":179339087,"data_compressed":826992,"data_compressed_allocated":10309632,"data_compressed_original":178457415,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1294336,"data_stored":8285980,"data_compressed":291002,"data_compressed_allocated":1032192,"data_compressed_original":8134656,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11329536,"data_stored":167819447,"data_compressed":1158180,"data_compressed_allocated":10244096,"data_compressed_original":166969344,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:32:47.978 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [2, 6] 2024-09-12T09:32:47.978 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1', '2', '6'] 2024-09-12T09:32:47.978 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1 2 6 2024-09-12T09:32:47.979 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:47.977+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:47.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:48.006 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:48.027 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:48.045 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:48.045 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:48.063 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:48.080 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:48.079+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:48.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:48.105 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:48.105 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:48.127 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:48.127 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:48.127 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:48.128 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:48.128 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:48.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:48.152 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:48.156 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:48.156 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:48.181 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:48.177+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:48.267 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:48.268 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:48.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:48.283 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:48.279+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:48.383 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:48.379+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 145.566719702, 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:32:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:32:48.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 145.219189814, 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:32:48.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:32:48.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 145.21673590399999, 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:32:48.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 145.214626017, 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:32:48.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:32:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 116.718879372, 2024-09-12T09:32:48.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:48.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:32:48.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:32:48.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:48.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.491104856000007, 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:32:48.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:32:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.491044294000005, 2024-09-12T09:32:48.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:32:48.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:32:48.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:32:48.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490980016999998, 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:32:48.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:32:48.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:32:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490912498, 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:32:48.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490807872000005, 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:32:48.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490744340000006, 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:32:48.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:32:48.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490706399999993, 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:32:48.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490677880000007, 2024-09-12T09:32:48.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:48.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:32:48.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490628376999993, 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:32:48.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490583987999997, 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:32:48.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490522104999997, 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:32:48.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490474591999998, 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:32:48.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:32:48.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:32:48.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490446321999997, 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490418769000001, 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:32:48.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:32:48.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.490387986000002, 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:32:48.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:32:48.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:48.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:48.548 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:48.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:48.550 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:48.547+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:48.650 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:48.645+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:48.721 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:48.722 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:48.733 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:32:48.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:48.738 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1 pg_upmap mapping to [2,6] 2024-09-12T09:32:48.764 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:48.763+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:48.853 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:48.849+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:48.865 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:48.865 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:48.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:48.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:48.953 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:48.949+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:49.007 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.570976837, 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.57095160599999, 2024-09-12T09:32:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.570913686, 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.570883685, 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.56284926399999, 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.562528391, 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.56205308600001, 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.562024338, 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.56147387199999, 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.56143105, 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.561385159, 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.560873601, 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.560825822, 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.56077017600001, 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.560738155, 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.560174006, 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.559780105, 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.559388407, 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 84.191522289000005, 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 56.333045493, 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:49.125 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:49.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:49.126 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:49.123+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:49.227 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:49.225+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:49.258 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:204395 3.1 3.e7005b41 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e191)", 2024-09-12T09:32:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:48.991788+0000", 2024-09-12T09:32:49.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.2674166, 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.26745984699999997, 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 204395 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991788+0000", 2024-09-12T09:32:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991788+0000", 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991791+0000", 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3530000000000002e-06 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991802+0000", 2024-09-12T09:32:49.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1557e-05 2024-09-12T09:32:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991833+0000", 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0849999999999998e-05 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.991848+0000", 2024-09-12T09:32:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.467e-05 2024-09-12T09:32:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.992027+0000", 2024-09-12T09:32:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000179068 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:48.992047+0000", 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0103e-05 2024-09-12T09:32:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-09-12T09:32:49.271 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:49.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:49.329 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:49.327+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:49.429 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:49.425+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:49.438 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:49.438 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:49.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:49.540 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:49.537+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:49.621 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.331375525, 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.871879296, 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49841509800001, 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49812202199999, 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49756009000001, 2024-09-12T09:32:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.497508727, 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.496935847, 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.496783548, 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.496426458, 2024-09-12T09:32:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49580148800001, 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.495750152, 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49495536200001, 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49488047200001, 2024-09-12T09:32:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.494828152, 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49477955399999, 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.49452508900001, 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.491070799, 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.43350225200001, 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.96223101499999, 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:202431 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.566943748, 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.971370387, 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 202431 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053485+0000", 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.716e-06 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053491+0000", 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.733e-06 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053504+0000", 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3002e-05 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053510+0000", 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0039999999999996e-06 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053627+0000", 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117124 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053640+0000", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2656999999999999e-05 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024816+0000", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97117616900000003 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024854+0000", 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7982000000000003e-05 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:49.738 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:49.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:49.740 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:49.733+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:49.841 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:49.839+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:49.915 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:49.915 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:49.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:49.976 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:49.973+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:50.043 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:50.037+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:50.046 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:50.047 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:50.061 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:50.144 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:50.143+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:50.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.816916658, 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.81682139599999, 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.81313677200001, 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:50.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:50.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.811331682, 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.81078500699999, 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:50.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:50.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:50.255 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.81001656399999, 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:50.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.809606196, 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:50.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:50.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.80946702, 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.266 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:50.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.80168820899999, 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:50.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.799324195, 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:50.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:50.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:50.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.78857230200001, 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:50.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:50.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.788375911, 2024-09-12T09:32:50.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:50.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:50.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:50.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 180.786881491, 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:50.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:50.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.08663863000001, 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:50.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.086616212, 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:50.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.08659307100001, 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:50.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 157.08656811200001, 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:50.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:50.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 147.04321659799999, 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:50.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.426738556, 2024-09-12T09:32:50.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:50.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.426639266, 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:50.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:50.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:50.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:50.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:50.336 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:50.337 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:50.337 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:50.338 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:50.338 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:50.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:50.339 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:50.339 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:50.335+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:50.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:50.345 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:50.364 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:50.365 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:50.384 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:50.402 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:50.422 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:50.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:50.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:50.437 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:50.433+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:50.469 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:50.469 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:50.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:50.576 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:50.573+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:50.638 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:50.633+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:50.647 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:50.647 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:50.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:50.740 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:50.739+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:50.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:32:50.841 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:50.839+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:50.845 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:50.846 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:50.864 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:50.942 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:50.937+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:50.999 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:50.999 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:51.043 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:51.043+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:51.144 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:51.141+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:51.179 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:51.180 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:51.246 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:51.243+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:51.346 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:51.341+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:51.357 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:51.357 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:51.447 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:51.445+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:51.507 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:51.507 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:51.571 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:51.571+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:51.649 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:51.645+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:51.649 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:51.650 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:51.750 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:51.749+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:51.760 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:32:51.847 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:51.847 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:51.851 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:51.851+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:51.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:51.953 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:51.949+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:52.032 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:52.033 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:52.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:52.054 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:52.053+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:52.154 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:52.149+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:52.171 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:52.171 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:52.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:52.256 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:52.253+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:52.313 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:52.313 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:52.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:52.371 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:52.371+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:52.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:52.457 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:52.457 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:52.458 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:52.453+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:52.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:52.559 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:52.553+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:52.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:52.590 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:52.590 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:52.607 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:52.628 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:32:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:32:52.646 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:52.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:52.661 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:52.657+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:52.670 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:52.670 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:52.692 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:52.693 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:52.696 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:52.696 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:52.761 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:52.759+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.356145979, 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:32:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:32:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:32:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.356120748, 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:32:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.356082828, 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.356052827, 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:32:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:32:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.34801840599999, 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:32:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:32:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.347697533, 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:32:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:32:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.34722222800001, 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:32:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:32:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.34719348, 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:32:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:32:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.34664301399999, 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:32:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.346600192, 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:32:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.346554301, 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:32:52.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.346042743, 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:32:52.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.345994964, 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.34593931800001, 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.345907297, 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:32:52.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:32:52.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.345343148, 2024-09-12T09:32:52.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:52.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:32:52.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.344949247, 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:32:52.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.344557549, 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:32:52.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.976691431000006, 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:32:52.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:52.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.118214635000001, 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:32:52.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:32:52.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:32:52.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:32:52.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:52.894 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:52.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:52.896 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:52.893+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:52.997 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:52.995+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:53.026 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:53.026 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:53.026 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:53.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:53.098 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:53.095+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:53.162 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.162 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:53.163 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:53.163 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:53.163 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:53.177 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:53.223 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:53.221+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:53.299 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:53.293+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:53.300 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.300 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:53.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:53.300 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:53.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 169.010969789, 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:53.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.55147356000001, 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:53.305 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:53.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17800936200001, 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:53.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:53.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:53.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17771628599999, 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:53.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:53.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17715435400001, 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:53.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:53.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.177102991, 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:53.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:53.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17653011100001, 2024-09-12T09:32:53.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:53.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:53.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.176377812, 2024-09-12T09:32:53.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:53.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:53.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.176020722, 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:53.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17539575199999, 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:53.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:53.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.175344416, 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:53.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17454962599999, 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:53.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:53.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17447473600001, 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:53.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.174422416, 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:53.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17437381799999, 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:53.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.17411935300001, 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:53.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:53.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.170665063, 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 150.11309651600001, 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:53.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:53.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 140.64182527899999, 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:53.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:53.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:202431 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.246538011999998, 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.971370387, 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 202431 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053485+0000", 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.716e-06 2024-09-12T09:32:53.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053491+0000", 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.733e-06 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053504+0000", 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3002e-05 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053510+0000", 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0039999999999996e-06 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053627+0000", 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117124 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053640+0000", 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2656999999999999e-05 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024816+0000", 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97117616900000003 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024854+0000", 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7982000000000003e-05 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.399 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:53.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:53.401 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:53.397+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:53.501 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:53.497+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:53.570 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:53.570 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:53.570 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:53.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:53.628 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:53.625+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:53.702 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:53.697+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:53.752 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:53.753 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:53.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:53.767 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:32:53.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:32:53.767 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-12T09:32:53.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-12T09:32:53.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:32:53.811 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:53.811+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:53.904 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:53.901+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.54672477099999, 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:53.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:53.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.54662950900001, 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:53.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.542944885, 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:53.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:53.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:53.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.54113979499999, 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:53.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:53.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.54059312000001, 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:53.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:53.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:53.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:53.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:53.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.53982467700001, 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:53.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.53941430899999, 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:53.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:53.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:53.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.53927513299999, 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:53.999 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:54.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:54.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.53149632200001, 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:54.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.52913230799999, 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:54.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:54.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:54.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.518380415, 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:54.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:54.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.51818402399999, 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:54.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:54.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:54.024 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 184.51668960399999, 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:54.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:54.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.816446743, 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:54.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.81642432500001, 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.816401184, 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:54.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:54.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 160.816376225, 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:54.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:54.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 150.77302471100001, 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:54.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:54.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 150.15654666899999, 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:54.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 150.15644737900001, 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:54.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:54.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.064 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:54.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:54.066 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:54.063+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 196, 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "177484da-1bae-43aa-8977-8d19d52d569f", 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-12T09:29:02.966061+0000", 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-12T09:32:53.767457+0000", 2024-09-12T09:32:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-12T09:31:57.668482+0000", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-12T09:31:51.831699+0000", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-12T09:32:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-12T09:32:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:16.007145+0000", 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "17", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 7.690000057220459, 2024-09-12T09:32:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 7.690000057220459, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.12999999523162842, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:22.198256+0000", 2024-09-12T09:32:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "21", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 21, 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.75, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-12T09:29:32.744560+0000", 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-12T09:32:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-12T09:32:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "196", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 55, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 196, 2024-09-12T09:32:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-12T09:32:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-12T09:32:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2400000095367432, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2400000095367432, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-12T09:32:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.86274528503417969, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "37709081-75a3-4326-93b5-5598d715ea86", 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 140, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 141, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 381521457 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/381521457", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/381521457", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/381521457", 2024-09-12T09:32:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/381521457", 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d25b680f-1839-4b35-8ad0-ae6bed56608f", 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 179, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2164663219 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/2164663219", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/2164663219", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/2164663219", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/2164663219", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "e209e2d0-0699-4641-a01e-462f5d295d7e", 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 194, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 139751272 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/139751272", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/139751272", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/139751272", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/139751272", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f00610a7-05b5-479e-ac16-f9a2b8a4ec1e", 2024-09-12T09:32:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.86334228515625, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 142, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4287683774 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/4287683774", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/4287683774", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/4287683774", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/4287683774", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85af91a4-b30a-47d4-a8cf-4a40e0c18869", 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 193, 2024-09-12T09:32:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6826", 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6827", 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6828", 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6829", 2024-09-12T09:32:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6832", 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6833", 2024-09-12T09:32:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6830", 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6831", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1982719715 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6827/1982719715", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6829/1982719715", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6833/1982719715", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6831/1982719715", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "a11aa028-c5a6-4b28-ad9e-8a24028273cc", 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 155, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6800", 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6801", 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6802", 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6803", 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6806", 2024-09-12T09:32:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6807", 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6804", 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6805", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1321805872 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6801/1321805872", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6803/1321805872", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6807/1321805872", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6805/1321805872", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "78df53b3-cb81-420f-8657-bf2c51381f54", 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.0386199951171875, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 154, 2024-09-12T09:32:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6808", 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6809", 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6810", 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6811", 2024-09-12T09:32:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6814", 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6815", 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6812", 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6813", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3633441839 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6809/3633441839", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6811/3633441839", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6815/3633441839", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6813/3633441839", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "393284af-2f90-46be-99a0-15f9015c3e7f", 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 187, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-12T09:32:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6816", 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6817", 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6818", 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6819", 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6822", 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6823", 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6820", 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.115:6821", 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 767353918 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.115:6817/767353918", 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.115:6819/767353918", 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.115:6823/767353918", 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.115:6821/767353918", 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.165826+0000", 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 141 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.541881+0000", 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.550164+0000", 2024-09-12T09:32:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.611505+0000", 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-12T09:32:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.331875+0000", 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.389036+0000", 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:12.613814+0000", 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-12T09:29:13.178370+0000", 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1", 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.e", 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.12", 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.f", 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 2 2024-09-12T09:32:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.3", 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.10", 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.17", 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: 3 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-12T09:32:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 17, 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 39 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 17, 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-12T09:32:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:32:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:54.207 INFO:tasks.thrashosds.thrasher:j is {'epoch': 196, 'fsid': '177484da-1bae-43aa-8977-8d19d52d569f', 'created': '2024-09-12T09:29:02.966061+0000', 'modified': '2024-09-12T09:32:53.767457+0000', 'last_up_change': '2024-09-12T09:31:57.668482+0000', 'last_in_change': '2024-09-12T09:31:51.831699+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T09:29:16.007145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-12T09:29:22.198256+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-12T09:29:32.744560+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 55, 'snap_epoch': 196, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8627452850341797, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '37709081-75a3-4326-93b5-5598d715ea86', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 140, 'down_at': 141, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 381521457}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 381521457}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 381521457}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 381521457}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 381521457}]}, 'public_addr': '172.21.15.7:6817/381521457', 'cluster_addr': '172.21.15.7:6819/381521457', 'heartbeat_back_addr': '172.21.15.7:6823/381521457', 'heartbeat_front_addr': '172.21.15.7:6821/381521457', 'state': ['exists']}, {'osd': 1, 'uuid': 'd25b680f-1839-4b35-8ad0-ae6bed56608f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 179, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 2164663219}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 2164663219}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 2164663219}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 2164663219}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 2164663219}]}, 'public_addr': '172.21.15.7:6825/2164663219', 'cluster_addr': '172.21.15.7:6827/2164663219', 'heartbeat_back_addr': '172.21.15.7:6831/2164663219', 'heartbeat_front_addr': '172.21.15.7:6829/2164663219', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e209e2d0-0699-4641-a01e-462f5d295d7e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 194, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 139751272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 139751272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 139751272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 139751272}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 139751272}]}, 'public_addr': '172.21.15.7:6801/139751272', 'cluster_addr': '172.21.15.7:6803/139751272', 'heartbeat_back_addr': '172.21.15.7:6807/139751272', 'heartbeat_front_addr': '172.21.15.7:6805/139751272', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f00610a7-05b5-479e-ac16-f9a2b8a4ec1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.86334228515625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 4287683774}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 4287683774}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 4287683774}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 4287683774}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 4287683774}]}, 'public_addr': '172.21.15.7:6809/4287683774', 'cluster_addr': '172.21.15.7:6811/4287683774', 'heartbeat_back_addr': '172.21.15.7:6815/4287683774', 'heartbeat_front_addr': '172.21.15.7:6813/4287683774', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85af91a4-b30a-47d4-a8cf-4a40e0c18869', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 193, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 1982719715}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 1982719715}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6832', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6833', 'nonce': 1982719715}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 1982719715}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 1982719715}]}, 'public_addr': '172.21.15.115:6827/1982719715', 'cluster_addr': '172.21.15.115:6829/1982719715', 'heartbeat_back_addr': '172.21.15.115:6833/1982719715', 'heartbeat_front_addr': '172.21.15.115:6831/1982719715', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'a11aa028-c5a6-4b28-ad9e-8a24028273cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 155, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 1321805872}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 1321805872}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 1321805872}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 1321805872}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 1321805872}]}, 'public_addr': '172.21.15.115:6801/1321805872', 'cluster_addr': '172.21.15.115:6803/1321805872', 'heartbeat_back_addr': '172.21.15.115:6807/1321805872', 'heartbeat_front_addr': '172.21.15.115:6805/1321805872', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78df53b3-cb81-420f-8657-bf2c51381f54', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0386199951171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 154, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 3633441839}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 3633441839}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 3633441839}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 3633441839}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 3633441839}]}, 'public_addr': '172.21.15.115:6809/3633441839', 'cluster_addr': '172.21.15.115:6811/3633441839', 'heartbeat_back_addr': '172.21.15.115:6815/3633441839', 'heartbeat_front_addr': '172.21.15.115:6813/3633441839', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '393284af-2f90-46be-99a0-15f9015c3e7f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 187, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 767353918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 767353918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 767353918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 767353918}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 767353918}]}, 'public_addr': '172.21.15.115:6817/767353918', 'cluster_addr': '172.21.15.115:6819/767353918', 'heartbeat_back_addr': '172.21.15.115:6823/767353918', 'heartbeat_front_addr': '172.21.15.115:6821/767353918', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.165826+0000', 'dead_epoch': 141}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.541881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.550164+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.611505+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.331875+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.389036+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:12.613814+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-12T09:29:13.178370+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [2, 6]}, {'pgid': '3.e', 'osds': [5, 0]}, {'pgid': '3.12', 'osds': [7, 2]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.3', 'osds': [4, 6]}, {'pgid': '3.b', 'osds': [3, 4]}, {'pgid': '3.10', 'osds': [6, 5]}, {'pgid': '3.17', 'osds': [5, 3]}, {'pgid': '3.18', 'osds': [3, 1]}, {'pgid': '3.1b', 'osds': [4, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 17, 'length': 39}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 17, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-12T09:32:54.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:32:54.208 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:54.201+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:54.240 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:54.240 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:54.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:54.307 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:54.307+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:54.408 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:54.407+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:54.424 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:54.424 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:54.424 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:54.439 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:54.509 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:54.505+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:54.563 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806941965999997, 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806891152999995, 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:54.569 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:54.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806854489000003, 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:54.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:54.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806795156000007, 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:54.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806765147999997, 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:54.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806746313000005, 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806719595000004, 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:54.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:54.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806698828999998, 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:54.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:54.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:54.605 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806679114000005, 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:54.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806648917999993, 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:54.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806618349999994, 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:54.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806599547000005, 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:54.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:54.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:54.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806550453, 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:54.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806515958000006, 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:54.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:54.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806483180000001, 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:54.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806448048999997, 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:54.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806405725000005, 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:54.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806370217999998, 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.648 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:54.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806335947999997, 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:54.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.806282611, 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:54.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:54.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:54.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:54.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:54.662 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:32:54.664 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:32:54.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:54.665 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:54.661+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:54.666 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-12T09:32:54.029949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":965485733,"num_objects":958,"num_object_clones":193,"num_object_copies":1916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":138,"num_objects_misplaced":190,"num_objects_unfound":0,"num_objects_dirty":958,"num_whiteouts":486,"num_read":252544,"num_read_kb":1580372,"num_write":22619,"num_write_kb":854048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4356,"num_bytes_recovered":835969734,"num_keys_recovered":715,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18938,"ondisk_log_size":18938,"up":69,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443512,"kb_used_data":74632,"kb_used_omap":12,"kb_used_meta":368691,"kb_avail":754531208,"statfs":{"total":773094113280,"available":772639956992,"internally_reserved":0,"allocated":76423168,"data_stored":879539677,"data_compressed":5970378,"data_compressed_allocated":52457472,"data_compressed_original":866378522,"omap_allocated":12752,"internal_metadata":377540144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-114708664,"num_objects":-79,"num_object_clones":-27,"num_object_copies":-158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":-20,"num_objects_unfound":0,"num_objects_dirty":-79,"num_whiteouts":-13,"num_read":944,"num_read_kb":802,"num_write":135,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001370"},"pg_stats":[{"pgid":"3.1f","version":"191'744","reported_seq":6396,"reported_epoch":193,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:50.806028+0000","last_change":"2024-09-12T09:31:58.689191+0000","last_active":"2024-09-12T09:32:50.806028+0000","last_peered":"2024-09-12T09:32:50.806028+0000","last_clean":"2024-09-12T09:31:51.668701+0000","last_became_active":"2024-09-12T09:31:58.689191+0000","last_became_peered":"2024-09-12T09:31:58.689191+0000","last_unstale":"2024-09-12T09:32:50.806028+0000","last_undegraded":"2024-09-12T09:31:58.686707+0000","last_fullsized":"2024-09-12T09:31:58.686488+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":37,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:42.259433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028210679999999999,"stat_sum":{"num_bytes":77170787,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":8213,"num_read_kb":95267,"num_write":948,"num_write_kb":47653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":37755293,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":44}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"191'720","reported_seq":6181,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.772980+0000","last_change":"2024-09-12T09:32:48.746600+0000","last_active":"2024-09-12T09:32:50.772980+0000","last_peered":"2024-09-12T09:32:50.772980+0000","last_clean":"2024-09-12T09:32:50.772980+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:32:50.772980+0000","last_undegraded":"2024-09-12T09:32:50.772980+0000","last_fullsized":"2024-09-12T09:32:50.772980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:36:29.071014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073534239999999999,"stat_sum":{"num_bytes":31784598,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":8676,"num_read_kb":41264,"num_write":826,"num_write_kb":38763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.1d","version":"191'515","reported_seq":8122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805692+0000","last_change":"2024-09-12T09:32:48.779299+0000","last_active":"2024-09-12T09:32:50.805692+0000","last_peered":"2024-09-12T09:32:50.805692+0000","last_clean":"2024-09-12T09:32:50.805692+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:32:50.805692+0000","last_undegraded":"2024-09-12T09:32:50.805692+0000","last_fullsized":"2024-09-12T09:32:50.805692+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:31:36.068487+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0096784269999999999,"stat_sum":{"num_bytes":37230621,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":9121,"num_read_kb":78651,"num_write":487,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.1c","version":"191'701","reported_seq":6958,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805687+0000","last_change":"2024-09-12T09:32:48.771480+0000","last_active":"2024-09-12T09:32:50.805687+0000","last_peered":"2024-09-12T09:32:50.805687+0000","last_clean":"2024-09-12T09:32:50.805687+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:32:50.805687+0000","last_undegraded":"2024-09-12T09:32:50.805687+0000","last_fullsized":"2024-09-12T09:32:50.805687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:42.043494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003221195,"stat_sum":{"num_bytes":19732589,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":8264,"num_read_kb":52460,"num_write":768,"num_write_kb":16449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.1b","version":"194'377","reported_seq":6503,"reported_epoch":194,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:51.772010+0000","last_change":"2024-09-12T09:32:51.772010+0000","last_active":"2024-09-12T09:32:51.772010+0000","last_peered":"2024-09-12T09:32:51.772010+0000","last_clean":"2024-09-12T09:32:51.772010+0000","last_became_active":"2024-09-12T09:31:59.687414+0000","last_became_peered":"2024-09-12T09:31:59.687414+0000","last_unstale":"2024-09-12T09:32:51.772010+0000","last_undegraded":"2024-09-12T09:32:51.772010+0000","last_fullsized":"2024-09-12T09:32:51.772010+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:19.325162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059235360000000001,"stat_sum":{"num_bytes":9481915,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":6296,"num_read_kb":20885,"num_write":365,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[{"shards":"4,7","objects":23}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.1a","version":"189'579","reported_seq":9432,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.955167+0000","last_change":"2024-09-12T09:32:51.770764+0000","last_active":"2024-09-12T09:32:51.955167+0000","last_peered":"2024-09-12T09:32:51.955167+0000","last_clean":"2024-09-12T09:32:51.955167+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:32:51.955167+0000","last_undegraded":"2024-09-12T09:32:51.955167+0000","last_fullsized":"2024-09-12T09:32:51.955167+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:02:39.319295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048328160000000002,"stat_sum":{"num_bytes":8311250,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":9018,"num_read_kb":32059,"num_write":615,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.19","version":"191'408","reported_seq":3815,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.757953+0000","last_change":"2024-09-12T09:32:48.742922+0000","last_active":"2024-09-12T09:32:50.757953+0000","last_peered":"2024-09-12T09:32:50.757953+0000","last_clean":"2024-09-12T09:32:50.757953+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:32:50.757953+0000","last_undegraded":"2024-09-12T09:32:50.757953+0000","last_fullsized":"2024-09-12T09:32:50.757953+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:53.142170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085511500000000004,"stat_sum":{"num_bytes":13597616,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":4048,"num_read_kb":15043,"num_write":460,"num_write_kb":6310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.18","version":"191'465","reported_seq":4126,"reported_epoch":193,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:50.772494+0000","last_change":"2024-09-12T09:32:48.745022+0000","last_active":"2024-09-12T09:32:50.772494+0000","last_peered":"2024-09-12T09:32:50.772494+0000","last_clean":"2024-09-12T09:32:50.772494+0000","last_became_active":"2024-09-12T09:31:59.685845+0000","last_became_peered":"2024-09-12T09:31:59.685845+0000","last_unstale":"2024-09-12T09:32:50.772494+0000","last_undegraded":"2024-09-12T09:32:50.772494+0000","last_fullsized":"2024-09-12T09:32:50.772494+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:24.174585+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0065855740000000003,"stat_sum":{"num_bytes":5069729,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":2635,"num_read_kb":10192,"num_write":490,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":23}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.17","version":"191'617","reported_seq":4982,"reported_epoch":193,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:50.806063+0000","last_change":"2024-09-12T09:32:48.789541+0000","last_active":"2024-09-12T09:32:50.806063+0000","last_peered":"2024-09-12T09:32:50.806063+0000","last_clean":"2024-09-12T09:32:50.806063+0000","last_became_active":"2024-09-12T09:31:59.686284+0000","last_became_peered":"2024-09-12T09:31:59.686284+0000","last_unstale":"2024-09-12T09:32:50.806063+0000","last_undegraded":"2024-09-12T09:32:50.806063+0000","last_fullsized":"2024-09-12T09:32:50.806063+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:07:28.029747+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018767385000000001,"stat_sum":{"num_bytes":25828235,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":6896,"num_read_kb":53657,"num_write":713,"num_write_kb":39724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":14}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.16","version":"191'525","reported_seq":15813,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.806014+0000","last_change":"2024-09-12T09:32:48.781026+0000","last_active":"2024-09-12T09:32:50.806014+0000","last_peered":"2024-09-12T09:32:50.806014+0000","last_clean":"2024-09-12T09:32:50.806014+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:32:50.806014+0000","last_undegraded":"2024-09-12T09:32:50.806014+0000","last_fullsized":"2024-09-12T09:32:50.806014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:53.202370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01044515,"stat_sum":{"num_bytes":39701644,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":15916,"num_read_kb":96835,"num_write":566,"num_write_kb":40304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.15","version":"191'792","reported_seq":8697,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.772087+0000","last_change":"2024-09-12T09:32:51.772962+0000","last_active":"2024-09-12T09:32:51.772087+0000","last_peered":"2024-09-12T09:32:51.772087+0000","last_clean":"2024-09-12T09:32:51.772087+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:32:51.772087+0000","last_undegraded":"2024-09-12T09:32:51.772087+0000","last_fullsized":"2024-09-12T09:32:51.772087+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:06.115434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057135420000000003,"stat_sum":{"num_bytes":63988982,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":9168,"num_read_kb":102695,"num_write":1015,"num_write_kb":47722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.14","version":"194'816","reported_seq":12275,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.922573+0000","last_change":"2024-09-12T09:32:51.775630+0000","last_active":"2024-09-12T09:32:51.922573+0000","last_peered":"2024-09-12T09:32:51.922573+0000","last_clean":"2024-09-12T09:32:51.922573+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:32:51.922573+0000","last_undegraded":"2024-09-12T09:32:51.922573+0000","last_fullsized":"2024-09-12T09:32:51.922573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:20:21.591078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083672239999999995,"stat_sum":{"num_bytes":16924694,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":13288,"num_read_kb":53936,"num_write":1131,"num_write_kb":31339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.768095+0000","last_change":"2024-09-12T09:32:22.911749+0000","last_active":"2024-09-12T09:32:51.768095+0000","last_peered":"2024-09-12T09:32:51.768095+0000","last_clean":"2024-09-12T09:32:51.768095+0000","last_became_active":"2024-09-12T09:32:22.911415+0000","last_became_peered":"2024-09-12T09:32:22.911415+0000","last_unstale":"2024-09-12T09:32:51.768095+0000","last_undegraded":"2024-09-12T09:32:51.768095+0000","last_fullsized":"2024-09-12T09:32:51.768095+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:37.113174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"191'367","reported_seq":6993,"reported_epoch":193,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:50.758263+0000","last_change":"2024-09-12T09:31:58.680982+0000","last_active":"2024-09-12T09:32:50.758263+0000","last_peered":"2024-09-12T09:32:50.758263+0000","last_clean":"2024-09-12T09:31:51.383028+0000","last_became_active":"2024-09-12T09:31:58.680982+0000","last_became_peered":"2024-09-12T09:31:58.680982+0000","last_unstale":"2024-09-12T09:32:50.758263+0000","last_undegraded":"2024-09-12T09:31:58.679925+0000","last_fullsized":"2024-09-12T09:31:58.679826+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":37,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:15:37.208370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025279059999999999,"stat_sum":{"num_bytes":51026203,"num_objects":41,"num_object_clones":15,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":20,"num_read":6905,"num_read_kb":29670,"num_write":311,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":147,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.758114+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:32:50.758114+0000","last_peered":"2024-09-12T09:32:50.758114+0000","last_clean":"2024-09-12T09:32:50.758114+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:32:50.758114+0000","last_undegraded":"2024-09-12T09:32:50.758114+0000","last_fullsized":"2024-09-12T09:32:50.758114+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"194'421","reported_seq":4059,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.774526+0000","last_change":"2024-09-12T09:32:51.774526+0000","last_active":"2024-09-12T09:32:51.774526+0000","last_peered":"2024-09-12T09:32:51.774526+0000","last_clean":"2024-09-12T09:32:51.774526+0000","last_became_active":"2024-09-12T09:30:33.671972+0000","last_became_peered":"2024-09-12T09:30:33.671972+0000","last_unstale":"2024-09-12T09:32:51.774526+0000","last_undegraded":"2024-09-12T09:32:51.774526+0000","last_fullsized":"2024-09-12T09:32:51.774526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:39.079322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069601619999999998,"stat_sum":{"num_bytes":13349058,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":3932,"num_read_kb":30622,"num_write":462,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"2.5","version":"0'0","reported_seq":275,"reported_epoch":194,"state":"active+undersized","last_fresh":"2024-09-12T09:32:51.766910+0000","last_change":"2024-09-12T09:31:58.689633+0000","last_active":"2024-09-12T09:32:51.766910+0000","last_peered":"2024-09-12T09:32:51.766910+0000","last_clean":"2024-09-12T09:31:56.850902+0000","last_became_active":"2024-09-12T09:31:58.689633+0000","last_became_peered":"2024-09-12T09:31:58.689633+0000","last_unstale":"2024-09-12T09:32:51.766910+0000","last_undegraded":"2024-09-12T09:32:51.766910+0000","last_fullsized":"2024-09-12T09:31:58.684053+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:23.955347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"191'1131","reported_seq":8046,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.758011+0000","last_change":"2024-09-12T09:32:48.743916+0000","last_active":"2024-09-12T09:32:50.758011+0000","last_peered":"2024-09-12T09:32:50.758011+0000","last_clean":"2024-09-12T09:32:50.758011+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:32:50.758011+0000","last_undegraded":"2024-09-12T09:32:50.758011+0000","last_fullsized":"2024-09-12T09:32:50.758011+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:55:48.986211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00143654,"stat_sum":{"num_bytes":2991471,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":11145,"num_read_kb":45471,"num_write":1742,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"2.4","version":"0'0","reported_seq":156,"reported_epoch":193,"state":"active+undersized","last_fresh":"2024-09-12T09:32:50.758145+0000","last_change":"2024-09-12T09:31:58.680941+0000","last_active":"2024-09-12T09:32:50.758145+0000","last_peered":"2024-09-12T09:32:50.758145+0000","last_clean":"2024-09-12T09:31:56.702759+0000","last_became_active":"2024-09-12T09:31:58.680941+0000","last_became_peered":"2024-09-12T09:31:58.680941+0000","last_unstale":"2024-09-12T09:32:50.758145+0000","last_undegraded":"2024-09-12T09:32:50.758145+0000","last_fullsized":"2024-09-12T09:31:58.679822+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:32.828980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"191'260","reported_seq":14065,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805842+0000","last_change":"2024-09-12T09:32:48.786383+0000","last_active":"2024-09-12T09:32:50.805842+0000","last_peered":"2024-09-12T09:32:50.805842+0000","last_clean":"2024-09-12T09:32:50.805842+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:32:50.805842+0000","last_undegraded":"2024-09-12T09:32:50.805842+0000","last_fullsized":"2024-09-12T09:32:50.805842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.307049+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015968691,"stat_sum":{"num_bytes":25542884,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":18,"num_read":13557,"num_read_kb":60534,"num_write":214,"num_write_kb":14069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.3","version":"194'1091","reported_seq":6244,"reported_epoch":194,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:51.940997+0000","last_change":"2024-09-12T09:32:51.772263+0000","last_active":"2024-09-12T09:32:51.940997+0000","last_peered":"2024-09-12T09:32:51.940997+0000","last_clean":"2024-09-12T09:32:51.940997+0000","last_became_active":"2024-09-12T09:31:59.688563+0000","last_became_peered":"2024-09-12T09:31:59.688563+0000","last_unstale":"2024-09-12T09:32:51.940997+0000","last_undegraded":"2024-09-12T09:32:51.940997+0000","last_fullsized":"2024-09-12T09:32:51.940997+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:43.978327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059870360000000003,"stat_sum":{"num_bytes":16118548,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":7837,"num_read_kb":25123,"num_write":1457,"num_write_kb":11307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":35}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"2.2","version":"21'2","reported_seq":406,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805459+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:32:50.805459+0000","last_peered":"2024-09-12T09:32:50.805459+0000","last_clean":"2024-09-12T09:32:50.805459+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:32:50.805459+0000","last_undegraded":"2024-09-12T09:32:50.805459+0000","last_fullsized":"2024-09-12T09:32:50.805459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"191'392","reported_seq":5076,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.757877+0000","last_change":"2024-09-12T09:32:48.746859+0000","last_active":"2024-09-12T09:32:50.757877+0000","last_peered":"2024-09-12T09:32:50.757877+0000","last_clean":"2024-09-12T09:32:50.757877+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:32:50.757877+0000","last_undegraded":"2024-09-12T09:32:50.757877+0000","last_fullsized":"2024-09-12T09:32:50.757877+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:17:57.993109+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0047392479999999997,"stat_sum":{"num_bytes":23481704,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":4684,"num_read_kb":38015,"num_write":335,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.756664+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:32:50.756664+0000","last_peered":"2024-09-12T09:32:50.756664+0000","last_clean":"2024-09-12T09:32:50.756664+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:32:50.756664+0000","last_undegraded":"2024-09-12T09:32:50.756664+0000","last_fullsized":"2024-09-12T09:32:50.756664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"179'488","reported_seq":13233,"reported_epoch":189,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:47.016243+0000","last_change":"2024-09-12T09:32:46.733218+0000","last_active":"2024-09-12T09:32:47.016243+0000","last_peered":"2024-09-12T09:32:47.016243+0000","last_clean":"2024-09-12T09:32:47.016243+0000","last_became_active":"2024-09-12T09:31:59.684097+0000","last_became_peered":"2024-09-12T09:31:59.684097+0000","last_unstale":"2024-09-12T09:32:47.016243+0000","last_undegraded":"2024-09-12T09:32:47.016243+0000","last_fullsized":"2024-09-12T09:32:47.016243+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:32.085976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001941538,"stat_sum":{"num_bytes":36625257,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":13928,"num_read_kb":39079,"num_write":516,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":10010383,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":27}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"15","length":"23"}]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.765350+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:32:51.765350+0000","last_peered":"2024-09-12T09:32:51.765350+0000","last_clean":"2024-09-12T09:32:51.765350+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:32:51.765350+0000","last_undegraded":"2024-09-12T09:32:51.765350+0000","last_fullsized":"2024-09-12T09:32:51.765350+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"187'375","reported_seq":5746,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805432+0000","last_change":"2024-09-12T09:32:48.770220+0000","last_active":"2024-09-12T09:32:50.805432+0000","last_peered":"2024-09-12T09:32:50.805432+0000","last_clean":"2024-09-12T09:32:50.805432+0000","last_became_active":"2024-09-12T09:32:13.696998+0000","last_became_peered":"2024-09-12T09:32:13.696998+0000","last_unstale":"2024-09-12T09:32:50.805432+0000","last_undegraded":"2024-09-12T09:32:50.805432+0000","last_fullsized":"2024-09-12T09:32:50.805432+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:55.452921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021300339999999998,"stat_sum":{"num_bytes":29564344,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":6310,"num_read_kb":47019,"num_write":368,"num_write_kb":14935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":9357778,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"2.3","version":"19'1","reported_seq":365,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805559+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:32:50.805559+0000","last_peered":"2024-09-12T09:32:50.805559+0000","last_clean":"2024-09-12T09:32:50.805559+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:32:50.805559+0000","last_undegraded":"2024-09-12T09:32:50.805559+0000","last_fullsized":"2024-09-12T09:32:50.805559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":305,"reported_epoch":194,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:51.764826+0000","last_change":"2024-09-12T09:31:58.683002+0000","last_active":"2024-09-12T09:32:51.764826+0000","last_peered":"2024-09-12T09:32:51.764826+0000","last_clean":"2024-09-12T09:31:56.850615+0000","last_became_active":"2024-09-12T09:31:58.683002+0000","last_became_peered":"2024-09-12T09:31:58.683002+0000","last_unstale":"2024-09-12T09:32:51.764826+0000","last_undegraded":"2024-09-12T09:31:58.681286+0000","last_fullsized":"2024-09-12T09:31:58.681061+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:58:41.462657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"193'1262","reported_seq":6253,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.842979+0000","last_change":"2024-09-12T09:32:48.745749+0000","last_active":"2024-09-12T09:32:50.842979+0000","last_peered":"2024-09-12T09:32:50.842979+0000","last_clean":"2024-09-12T09:32:50.842979+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:32:50.842979+0000","last_undegraded":"2024-09-12T09:32:50.842979+0000","last_fullsized":"2024-09-12T09:32:50.842979+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:26:03.123850+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0074198930000000003,"stat_sum":{"num_bytes":35392808,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":11149,"num_read_kb":56967,"num_write":1908,"num_write_kb":24771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.9","version":"194'715","reported_seq":4999,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.769354+0000","last_change":"2024-09-12T09:32:51.769354+0000","last_active":"2024-09-12T09:32:51.769354+0000","last_peered":"2024-09-12T09:32:51.769354+0000","last_clean":"2024-09-12T09:32:51.769354+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:32:51.769354+0000","last_undegraded":"2024-09-12T09:32:51.769354+0000","last_fullsized":"2024-09-12T09:32:51.769354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:31:23.897660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040484120000000004,"stat_sum":{"num_bytes":8758125,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":6054,"num_read_kb":30020,"num_write":913,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.a","version":"194'457","reported_seq":5765,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.776014+0000","last_change":"2024-09-12T09:32:51.776014+0000","last_active":"2024-09-12T09:32:51.776014+0000","last_peered":"2024-09-12T09:32:51.776014+0000","last_clean":"2024-09-12T09:32:51.776014+0000","last_became_active":"2024-09-12T09:32:22.909677+0000","last_became_peered":"2024-09-12T09:32:22.909677+0000","last_unstale":"2024-09-12T09:32:51.776014+0000","last_undegraded":"2024-09-12T09:32:51.776014+0000","last_fullsized":"2024-09-12T09:32:51.776014+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:44.279552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082971580000000007,"stat_sum":{"num_bytes":17487663,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":5042,"num_read_kb":17281,"num_write":525,"num_write_kb":10441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.b","version":"191'469","reported_seq":6616,"reported_epoch":193,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:50.772732+0000","last_change":"2024-09-12T09:32:48.754565+0000","last_active":"2024-09-12T09:32:50.772732+0000","last_peered":"2024-09-12T09:32:50.772732+0000","last_clean":"2024-09-12T09:32:50.772732+0000","last_became_active":"2024-09-12T09:31:59.685428+0000","last_became_peered":"2024-09-12T09:31:59.685428+0000","last_unstale":"2024-09-12T09:32:50.772732+0000","last_undegraded":"2024-09-12T09:32:50.772732+0000","last_fullsized":"2024-09-12T09:32:50.772732+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:34:55.053413+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016009310999999998,"stat_sum":{"num_bytes":60325389,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":6709,"num_read_kb":72316,"num_write":620,"num_write_kb":32938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":39}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.c","version":"191'448","reported_seq":4253,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-12T09:32:50.805843+0000","last_change":"2024-09-12T09:32:48.779956+0000","last_active":"2024-09-12T09:32:50.805843+0000","last_peered":"2024-09-12T09:32:50.805843+0000","last_clean":"2024-09-12T09:32:50.805843+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:32:50.805843+0000","last_undegraded":"2024-09-12T09:32:50.805843+0000","last_fullsized":"2024-09-12T09:32:50.805843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:48:48.709388+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010782138,"stat_sum":{"num_bytes":13480245,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":5343,"num_read_kb":44505,"num_write":423,"num_write_kb":19905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.d","version":"191'549","reported_seq":8945,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.770432+0000","last_change":"2024-09-12T09:32:51.770432+0000","last_active":"2024-09-12T09:32:51.770432+0000","last_peered":"2024-09-12T09:32:51.770432+0000","last_clean":"2024-09-12T09:32:51.770432+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:32:51.770432+0000","last_undegraded":"2024-09-12T09:32:51.770432+0000","last_fullsized":"2024-09-12T09:32:51.770432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:50:44.727410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041206439999999997,"stat_sum":{"num_bytes":19169678,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":9515,"num_read_kb":28899,"num_write":577,"num_write_kb":7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.e","version":"190'554","reported_seq":5963,"reported_epoch":193,"state":"active+undersized+degraded","last_fresh":"2024-09-12T09:32:50.805567+0000","last_change":"2024-09-12T09:31:58.688892+0000","last_active":"2024-09-12T09:32:50.805567+0000","last_peered":"2024-09-12T09:32:50.805567+0000","last_clean":"2024-09-12T09:31:56.715994+0000","last_became_active":"2024-09-12T09:31:58.688892+0000","last_became_peered":"2024-09-12T09:31:58.688892+0000","last_unstale":"2024-09-12T09:32:50.805567+0000","last_undegraded":"2024-09-12T09:31:58.686246+0000","last_fullsized":"2024-09-12T09:31:58.686019+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":37,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:33.433147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001396684,"stat_sum":{"num_bytes":67700443,"num_objects":51,"num_object_clones":16,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":27,"num_read":7112,"num_read_kb":94430,"num_write":623,"num_write_kb":45689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":119880838,"num_keys_recovered":83,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":51}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"194'643","reported_seq":7351,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.904553+0000","last_change":"2024-09-12T09:32:51.774471+0000","last_active":"2024-09-12T09:32:51.904553+0000","last_peered":"2024-09-12T09:32:51.904553+0000","last_clean":"2024-09-12T09:32:51.904553+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:32:51.904553+0000","last_undegraded":"2024-09-12T09:32:51.904553+0000","last_fullsized":"2024-09-12T09:32:51.904553+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:16:31.214547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060367789999999999,"stat_sum":{"num_bytes":46672051,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":7396,"num_read_kb":80041,"num_write":730,"num_write_kb":46022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.10","version":"184'657","reported_seq":3996,"reported_epoch":193,"state":"active+clean+remapped","last_fresh":"2024-09-12T09:32:50.755443+0000","last_change":"2024-09-12T09:32:48.748314+0000","last_active":"2024-09-12T09:32:50.755443+0000","last_peered":"2024-09-12T09:32:50.755443+0000","last_clean":"2024-09-12T09:32:50.755443+0000","last_became_active":"2024-09-12T09:31:59.687172+0000","last_became_peered":"2024-09-12T09:31:59.687172+0000","last_unstale":"2024-09-12T09:32:50.755443+0000","last_undegraded":"2024-09-12T09:32:50.755443+0000","last_fullsized":"2024-09-12T09:32:50.755443+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:46.526022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030584399999999999,"stat_sum":{"num_bytes":26015975,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":19,"num_read":3691,"num_read_kb":27710,"num_write":810,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[{"shards":"5,6","objects":29}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"13","length":"25"}]},{"pgid":"3.11","version":"193'656","reported_seq":3540,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.770401+0000","last_change":"2024-09-12T09:32:51.770401+0000","last_active":"2024-09-12T09:32:51.770401+0000","last_peered":"2024-09-12T09:32:51.770401+0000","last_clean":"2024-09-12T09:32:51.770401+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:32:51.770401+0000","last_undegraded":"2024-09-12T09:32:51.770401+0000","last_fullsized":"2024-09-12T09:32:51.770401+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:18:22.753748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049347210000000004,"stat_sum":{"num_bytes":27111970,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":5645,"num_read_kb":20017,"num_write":768,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.12","version":"191'422","reported_seq":13640,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.770716+0000","last_change":"2024-09-12T09:32:51.771821+0000","last_active":"2024-09-12T09:32:51.770716+0000","last_peered":"2024-09-12T09:32:51.770716+0000","last_clean":"2024-09-12T09:32:51.770716+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:32:51.770716+0000","last_undegraded":"2024-09-12T09:32:51.770716+0000","last_fullsized":"2024-09-12T09:32:51.770716+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:03:06.980013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005442517,"stat_sum":{"num_bytes":62108607,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":13361,"num_read_kb":129873,"num_write":515,"num_write_kb":65716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]},{"pgid":"3.13","version":"188'247","reported_seq":1885,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-12T09:32:51.772186+0000","last_change":"2024-09-12T09:32:51.772186+0000","last_active":"2024-09-12T09:32:51.772186+0000","last_peered":"2024-09-12T09:32:51.772186+0000","last_clean":"2024-09-12T09:32:51.772186+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:32:51.772186+0000","last_undegraded":"2024-09-12T09:32:51.772186+0000","last_fullsized":"2024-09-12T09:32:51.772186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:12:06.905771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057765109999999998,"stat_sum":{"num_bytes":33150263,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1624,"num_read_kb":9781,"num_write":315,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"26"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":964895346,"num_objects":954,"num_object_clones":193,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":136,"num_objects_misplaced":190,"num_objects_unfound":0,"num_objects_dirty":954,"num_whiteouts":486,"num_read":252478,"num_read_kb":1580317,"num_write":22516,"num_write_kb":852418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4340,"num_bytes_recovered":832427450,"num_keys_recovered":715,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61034496,"data_stored":897644170,"data_compressed":6580122,"data_compressed_allocated":54411264,"data_compressed_original":892852366,"omap_allocated":0,"internal_metadata":0},"log_size":18863,"ondisk_log_size":18863,"up":54,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":1,"acting":1,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70204,"kb_used_data":12648,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94301636,"statfs":{"total":96636764160,"available":96564875264,"internally_reserved":0,"allocated":12951552,"data_stored":159317595,"data_compressed":1030232,"data_compressed_allocated":9576448,"data_compressed_original":157343744,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55664,"kb_used_data":10580,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94316176,"statfs":{"total":96636764160,"available":96579764224,"internally_reserved":0,"allocated":10833920,"data_stored":116791596,"data_compressed":840176,"data_compressed_allocated":7528448,"data_compressed_original":114991078,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42816,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329024,"statfs":{"total":96636764160,"available":96592920576,"internally_reserved":0,"allocated":3387392,"data_stored":7285923,"data_compressed":258869,"data_compressed_allocated":839680,"data_compressed_original":6017024,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70328,"kb_used_data":12768,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94301512,"statfs":{"total":96636764160,"available":96564748288,"internally_reserved":0,"allocated":13074432,"data_stored":170270523,"data_compressed":734148,"data_compressed_allocated":9633792,"data_compressed_original":168284160,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60600,"kb_used_data":11364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311240,"statfs":{"total":96636764160,"available":96574709760,"internally_reserved":0,"allocated":11636736,"data_stored":131833118,"data_compressed":978364,"data_compressed_allocated":8486912,"data_compressed_original":130156340,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61376,"kb_used_data":7972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310464,"statfs":{"total":96636764160,"available":96573915136,"internally_reserved":0,"allocated":8163328,"data_stored":106993755,"data_compressed":415186,"data_compressed_allocated":5439488,"data_compressed_original":105508864,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35760,"kb_used_data":4572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336080,"statfs":{"total":96636764160,"available":96600145920,"internally_reserved":0,"allocated":4681728,"data_stored":23168605,"data_compressed":378501,"data_compressed_allocated":1945600,"data_compressed_original":21770240,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46764,"kb_used_data":11420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325076,"statfs":{"total":96636764160,"available":96588877824,"internally_reserved":0,"allocated":11694080,"data_stored":163878562,"data_compressed":1334902,"data_compressed_allocated":9007104,"data_compressed_original":162307072,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11812864,"data_stored":181540902,"data_compressed":1872412,"data_compressed_allocated":10399744,"data_compressed_original":180512892,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2367488,"data_stored":22036298,"data_compressed":378501,"data_compressed_allocated":1945600,"data_compressed_original":21770240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6361088,"data_stored":111960595,"data_compressed":485021,"data_compressed_allocated":5873664,"data_compressed_original":111549564,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9326592,"data_stored":130700830,"data_compressed":978364,"data_compressed_allocated":8486912,"data_compressed_original":130156340,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8499200,"data_stored":115646606,"data_compressed":840176,"data_compressed_allocated":7528448,"data_compressed_original":114991078,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10772480,"data_stored":169144397,"data_compressed":734148,"data_compressed_allocated":9633792,"data_compressed_original":168284160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":12258919,"data_compressed":328704,"data_compressed_allocated":1273856,"data_compressed_original":12057724,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10297344,"data_stored":154355623,"data_compressed":962796,"data_compressed_allocated":9269248,"data_compressed_original":153530368,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:32:54.668 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [0, 6] 2024-09-12T09:32:54.668 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '0', '6'] 2024-09-12T09:32:54.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 0 6 2024-09-12T09:32:54.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:54.764 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:54.761+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:54.783 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:32:54.847 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:54.848 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:54.848 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:54.848 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:54.864 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:54.863+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:54.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:54.874 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:54.874 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:54.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:54.913 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:54.913 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:54.934 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:54.954 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:54.955 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:54.965 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:54.961+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:54.972 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:54.989 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:54.989 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:54.993 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:54.993 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.038 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:55.066 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.063+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:55.167 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.167+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:55.217 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.218 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:55.279 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.279+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:55.366 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.366 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.369 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:55.367+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:55.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:55.470 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.467+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:55.552 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.552 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:55.570 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:55.569+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:55.672 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:55.671+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:55.795 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:55.795+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.2 pg_upmap mapping to [0,6] 2024-09-12T09:32:55.874 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.871+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:55.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:55.995 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:55.995+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:56.076 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:56.073+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.124 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:32:56.183 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:56.183+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:32:56.279 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:56.273+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:56.379 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:56.379+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:56.447 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.447 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:32:56.503 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:56.503+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:56.581 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:56.577+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:56.624 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.625 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:32:56.682 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:56.677+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:56.755 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.755 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:32:56.782 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:56.779+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:56.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:32:56.883 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:56.883+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:56.914 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:56.914 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:56.930 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:32:56.984 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:56.983+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:56.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:32:57.064 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:57.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:57.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:32:57.116 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:57.123 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:57.121+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:57.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:57.152 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:57.169 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:57.186 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:57.181+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:57.188 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4497.0:49738 3.f e50/23 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:head v 50'115, mlcod=49'113)", 2024-09-12T09:32:57.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 172.91447739700001, 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.073755786000000004, 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289052+0000", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.289055+0000", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318814+0000", 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029759397999999999 2024-09-12T09:32:57.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318858+0000", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3591000000000003e-05 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.318876+0000", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7978999999999998e-05 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319016+0000", 2024-09-12T09:32:57.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000140326 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.319068+0000", 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2203999999999997e-05 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362693+0000", 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043624893999999997 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:04.362808+0000", 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000114344 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:52581 3.e 3:73e7b21d:::image_client.0-clone11.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 168.45498116799999, 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95946498499999999, 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 52581 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748548+0000", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748550+0000", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748561+0000", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0467e-05 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748592+0000", 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1185e-05 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748607+0000", 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5293000000000001e-05 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748790+0000", 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018317499999999999 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:08.748822+0000", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1977999999999998e-05 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.707711+0000", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.95888856499999997 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:09.708013+0000", 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030205599999999998 2024-09-12T09:32:57.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58793 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 18284544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.08151697, 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070068006000000002, 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58793 2024-09-12T09:32:57.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122012+0000", 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122240+0000", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000227662 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122249+0000", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4769999999999992e-06 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122270+0000", 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0151000000000001e-05 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122284+0000", 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4224e-05 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191841+0000", 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069557169000000002 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.191958+0000", 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011709099999999999 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192080+0000", 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000122232 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58794 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13107200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:57.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.081223894, 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070391317999999994, 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58794 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122305+0000", 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122782+0000", 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00047698800000000002 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122789+0000", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8530000000000003e-06 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122810+0000", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0482000000000001e-05 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122824+0000", 2024-09-12T09:32:57.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4443e-05 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192144+0000", 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069319978000000004 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192202+0000", 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8360999999999999e-05 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192697+0000", 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049421299999999997 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58796 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13205504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.08066196199999, 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070316906999999998, 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58796 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122867+0000", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122869+0000", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122874+0000", 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6219999999999997e-06 2024-09-12T09:32:57.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122889+0000", 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4447000000000001e-05 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122900+0000", 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1158999999999999e-05 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192758+0000", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069858082000000002 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.192862+0000", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104095 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193184+0000", 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032216900000000003 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58797 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13303808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.08061059900001, 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070597798000000003, 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58797 2024-09-12T09:32:57.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.122919+0000", 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123437+0000", 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00051869700000000004 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123445+0000", 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.255e-06 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123458+0000", 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3552e-05 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123470+0000", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1804000000000001e-05 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193240+0000", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069770113999999994 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193311+0000", 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1465999999999996e-05 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193516+0000", 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020490999999999999 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58798 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13402112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.08003771899999, 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070462340999999998, 2024-09-12T09:32:57.232 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58798 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123491+0000", 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123585+0000", 2024-09-12T09:32:57.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3114000000000003e-05 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123592+0000", 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5820000000000004e-06 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123609+0000", 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7241999999999999e-05 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123623+0000", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3784000000000001e-05 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193556+0000", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069932862999999998 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193657+0000", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010127 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193954+0000", 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029648599999999999 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58799 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13500416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07988542000001, 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070649481, 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58799 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123644+0000", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123939+0000", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000295078 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123957+0000", 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8442000000000001e-05 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123972+0000", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5085999999999999e-05 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.123982+0000", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9909999999999997e-06 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.193992+0000", 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070009216999999999 2024-09-12T09:32:57.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194054+0000", 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2490999999999995e-05 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194293+0000", 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023917599999999999 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58800 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07952832999999, 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070835516000000001, 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58800 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124001+0000", 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124559+0000", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055762499999999996 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124566+0000", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1099999999999997e-06 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124585+0000", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9097e-05 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124599+0000", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4161999999999999e-05 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194366+0000", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069767440999999999 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194523+0000", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015644399999999999 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194836+0000", 2024-09-12T09:32:57.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031363700000000002 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58801 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13697024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07890336, 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070845748, 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58801 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124626+0000", 2024-09-12T09:32:57.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124628+0000", 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.885e-06 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124633+0000", 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124648+0000", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5440000000000001e-05 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124660+0000", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1352000000000001e-05 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194886+0000", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070226811 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.194977+0000", 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0773999999999995e-05 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195472+0000", 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00049440900000000002 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58802 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13795328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:57.252 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07885202400001, 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071226981999999994, 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58802 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.124677+0000", 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125401+0000", 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000723633 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125410+0000", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2099999999999999e-06 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125430+0000", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9621999999999999e-05 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125445+0000", 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5407999999999999e-05 2024-09-12T09:32:57.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195522+0000", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070077046000000004 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195638+0000", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011552699999999999 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195904+0000", 2024-09-12T09:32:57.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000266536 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58803 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13893632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.078057234, 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070802790000000004, 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.257 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58803 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125472+0000", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125473+0000", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.125e-06 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125478+0000", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125493+0000", 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5162e-05 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125504+0000", 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1391e-05 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.195955+0000", 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070451291999999999 2024-09-12T09:32:57.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196024+0000", 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8825999999999995e-05 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196275+0000", 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025049600000000001 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58805 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 13991936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07798234399999, 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071362717000000006, 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58805 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125547+0000", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125548+0000", 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.485e-06 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125554+0000", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2240000000000003e-06 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125569+0000", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5857000000000001e-05 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125581+0000", 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.207e-05 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196541+0000", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.070959747000000004 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196645+0000", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104157 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196910+0000", 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026417699999999999 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58806 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14090240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07793002400001, 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071736103999999995, 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58806 2024-09-12T09:32:57.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125599+0000", 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125601+0000", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3519999999999999e-06 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125607+0000", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125622+0000", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4746000000000001e-05 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125632+0000", 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0242e-05 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.196957+0000", 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071324464000000004 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197044+0000", 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7041999999999998e-05 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197335+0000", 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029133000000000002 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58807 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14188544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.077881426, 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072049815000000003, 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58807 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125648+0000", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125755+0000", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106943 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125760+0000", 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-12T09:32:57.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125772+0000", 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2208999999999999e-05 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125782+0000", 2024-09-12T09:32:57.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9129999999999998e-06 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197383+0000", 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.071601137999999995 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197461+0000", 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8008000000000003e-05 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197698+0000", 2024-09-12T09:32:57.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000236216 2024-09-12T09:32:57.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.275 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58808 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14286848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07762696099999, 2024-09-12T09:32:57.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.072210692000000007, 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58808 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:57.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.125902+0000", 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129157+0000", 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032546039999999999 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129168+0000", 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759e-05 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129192+0000", 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4272000000000001e-05 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129217+0000", 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5028000000000002e-05 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197733+0000", 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.068515777 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.197808+0000", 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4935e-05 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198113+0000", 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030531699999999999 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:58811 3.f 3:f273f2de:::rbd_data.119172280d2a.0000000000000000:c [read 14581760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.07417267100001, 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069984562, 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 58811 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129357+0000", 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129358+0000", 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.108e-06 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129363+0000", 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129379+0000", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6509000000000001e-05 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.129391+0000", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.189e-05 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.198973+0000", 2024-09-12T09:32:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.069581765000000004 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199073+0000", 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9734000000000004e-05 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.199341+0000", 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026857399999999999 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116931 3.14 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 154.016604124, 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.115163456, 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116931 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186925+0000", 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.186927+0000", 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6819999999999999e-06 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218626+0000", 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031698838999999999 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218659+0000", 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3801999999999999e-05 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218673+0000", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218801+0000", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012807699999999999 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.218960+0000", 2024-09-12T09:32:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015822199999999999 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.220469+0000", 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001509125 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.246257+0000", 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025788057999999999 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301912+0000", 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.055655537999999997 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.301991+0000", 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8697999999999998e-05 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.302089+0000", 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7496000000000003e-05 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:122458 3.14 3:2a23e97c:::rbd_data.11914da40b12.0000000000000055:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 144.545332887, 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0189399560000001, 2024-09-12T09:32:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 122458 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658196+0000", 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658198+0000", 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.013e-06 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658206+0000", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658219+0000", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658229+0000", 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5449999999999992e-06 2024-09-12T09:32:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658916+0000", 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00068749399999999997 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:32.658944+0000", 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7844999999999999e-05 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669089+0000", 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0101446460000001 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.669314+0000", 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000225572 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.670915+0000", 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001600588 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.676897+0000", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059818229999999998 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677041+0000", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014451000000000001 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677096+0000", 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5002e-05 2024-09-12T09:32:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:33.677136+0000", 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0034999999999999e-05 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:202431 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e177)", 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.15004562, 2024-09-12T09:32:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.971370387, 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 202431 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053484+0000", 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053485+0000", 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.716e-06 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053491+0000", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.733e-06 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053504+0000", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3002e-05 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053510+0000", 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0039999999999996e-06 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053627+0000", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000117124 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:35.053640+0000", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2656999999999999e-05 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024816+0000", 2024-09-12T09:32:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.97117616900000003 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:32:36.024854+0000", 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7982000000000003e-05 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.304 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:57.305 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:57.306 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:57.307 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:32:57.307 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:57.301+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:57.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:57.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:57.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:57.406 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:57.401+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:57.474 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:57.475 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:57.475 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:57.475 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:57.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:32:57.508 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:57.507+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:57.608 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:57.605+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:57.656 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:57.656 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:57.656 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:57.656 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:57.657 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:57.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:32:57.709 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:57.707+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:57.810 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:57.805+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:57.812 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:32:57.838 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9823 3.1f 3:fe75be6c:::rbd_data.11912e4e65e9.0000000000000081:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.42710433799999, 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.704282344, 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9823 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409969+0000", 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1399999999999997e-07 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409971+0000", 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.801e-06 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409977+0000", 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.409983+0000", 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-06 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410672+0000", 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000689706 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410683+0000", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1117999999999999e-05 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.113867+0000", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.703183263 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114251+0000", 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038434299999999999 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9853 3.1f 3:f9585bad:::rbd_data.11912e4e65e9.000000000000009f:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.42700907599999, 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70470139200000004, 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9853 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410064+0000", 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410065+0000", 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410067+0000", 2024-09-12T09:32:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410074+0000", 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8040000000000002e-06 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.410080+0000", 2024-09-12T09:32:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411678+0000", 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015980910000000001 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.411690+0000", 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2534999999999999e-05 2024-09-12T09:32:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114581+0000", 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.702890448 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114765+0000", 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018449199999999999 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:9924 3.1f 3:febcd7d4:::rbd_data.11912e4e65e9.00000000000000e6:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.423324452, 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.70134671999999998, 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9924 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413749+0000", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413750+0000", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413756+0000", 2024-09-12T09:32:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413769+0000", 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3467e-05 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.413782+0000", 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2741e-05 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415135+0000", 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013531680000000001 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415157+0000", 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2113e-05 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.114876+0000", 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69971925700000004 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115095+0000", 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021886299999999999 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10000 3.1f 3:ffbe264d:::rbd_data.11912e4e65e9.0000000000000132:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.421519362, 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69972894200000002, 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10000 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415554+0000", 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415557+0000", 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415567+0000", 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6220000000000004e-06 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415574+0000", 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1509999999999998e-06 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415643+0000", 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9323e-05 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.415657+0000", 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.366e-05 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115142+0000", 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.699485001 2024-09-12T09:32:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115283+0000", 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014076900000000001 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10039 3.1f 3:fc091c1c:::rbd_data.11912e4e65e9.0000000000000159:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.42097268699999, 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69956225400000005, 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10039 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416100+0000", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416102+0000", 2024-09-12T09:32:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8029999999999999e-06 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416112+0000", 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6760000000000002e-06 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416132+0000", 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0324999999999999e-05 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416149+0000", 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6529e-05 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416246+0000", 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7677999999999996e-05 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416259+0000", 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2984e-05 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115507+0000", 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69924735100000002 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115663+0000", 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015590800000000001 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10075 3.1f 3:fe131e99:::rbd_data.11912e4e65e9.000000000000017d:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.42020424399999, 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69907942599999995, 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10075 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416869+0000", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416870+0000", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416877+0000", 2024-09-12T09:32:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.046e-06 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416898+0000", 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0822000000000002e-05 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.416913+0000", 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4962e-05 2024-09-12T09:32:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417102+0000", 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018890899999999999 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417118+0000", 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5971e-05 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115730+0000", 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69861221900000003 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115948+0000", 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021811700000000001 2024-09-12T09:32:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10101 3.1f 3:f8474464:::rbd_data.11912e4e65e9.0000000000000197:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.419793876, 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.698866286, 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10101 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417279+0000", 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417281+0000", 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.33e-06 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417287+0000", 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2859999999999996e-06 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417307+0000", 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0018999999999999e-05 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417321+0000", 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4432e-05 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417542+0000", 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220595 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417555+0000", 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3609e-05 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.115998+0000", 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69844238599999997 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116145+0000", 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014762900000000001 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10121 3.1f 3:fc1ca568:::rbd_data.11912e4e65e9.00000000000001ab:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.4196547, 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69920736000000006, 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10121 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417418+0000", 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417420+0000", 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417442+0000", 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2708e-05 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417462+0000", 2024-09-12T09:32:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9108999999999999e-05 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417480+0000", 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8048000000000001e-05 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417740+0000", 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026004000000000001 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.417769+0000", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9734e-05 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116484+0000", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69871470899999999 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116626+0000", 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014169900000000001 2024-09-12T09:32:57.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10277 3.1f 3:fd6c2562:::rbd_data.11912e4e65e9.0000000000000247:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.41187588899999, 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.691692325, 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10277 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425197+0000", 2024-09-12T09:32:57.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425198+0000", 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.283e-06 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425204+0000", 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425223+0000", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8332e-05 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425236+0000", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3723000000000001e-05 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425473+0000", 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023660200000000001 2024-09-12T09:32:57.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.425487+0000", 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3919e-05 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116687+0000", 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.69119969599999997 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116890+0000", 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000202881 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10305 3.1f 3:fce47281:::rbd_data.11912e4e65e9.0000000000000263:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.40951187499999, 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68954919999999997, 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10305 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:57.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427561+0000", 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427562+0000", 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2099999999999996e-07 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427565+0000", 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0000000000000001e-06 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427574+0000", 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2909999999999996e-06 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427582+0000", 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.379e-06 2024-09-12T09:32:57.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427682+0000", 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100389 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.427695+0000", 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3471e-05 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.116962+0000", 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.68926653800000004 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117110+0000", 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014841100000000001 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10434 3.1f 3:fc31fc54:::rbd_data.11912e4e65e9.00000000000002e4:head [sparse-read 0~6308] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.398759982, 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67972727899999996, 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10434 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438313+0000", 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438314+0000", 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.35e-06 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438321+0000", 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7560000000000002e-06 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438345+0000", 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4124000000000001e-05 2024-09-12T09:32:57.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438364+0000", 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8465999999999999e-05 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438535+0000", 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017093300000000001 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438579+0000", 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4098999999999999e-05 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117165+0000", 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67858645500000003 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.117387+0000", 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022207600000000001 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118040+0000", 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00065302000000000003 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10447 3.1f 3:fe06451d:::rbd_data.11912e4e65e9.00000000000002f1:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.398563591, 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67975093399999997, 2024-09-12T09:32:57.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10447 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438509+0000", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438511+0000", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4759999999999999e-06 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438517+0000", 2024-09-12T09:32:57.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438537+0000", 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9915000000000002e-05 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438554+0000", 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6735000000000002e-05 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438642+0000", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8148999999999999e-05 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.438655+0000", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3123e-05 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118092+0000", 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67943649900000003 2024-09-12T09:32:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118260+0000", 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016861 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:10538 3.1f 3:fa559ef5:::rbd_data.11912e4e65e9.000000000000034c:head [sparse-read 0~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 188.397069171, 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67855747700000002, 2024-09-12T09:32:57.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10538 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440004+0000", 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440005+0000", 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.269e-06 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440012+0000", 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7440000000000001e-06 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440029+0000", 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7110999999999999e-05 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440043+0000", 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3909e-05 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440112+0000", 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8942999999999994e-05 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:49.440120+0000", 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6709999999999999e-06 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118366+0000", 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.67824624899999997 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:29:50.118561+0000", 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019558100000000001 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59516 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21168128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.69682631000001, 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074523105000000006, 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59516 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140247+0000", 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140250+0000", 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7130000000000001e-06 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140257+0000", 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6420000000000001e-06 2024-09-12T09:32:57.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140262+0000", 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214407+0000", 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074144598000000006 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214505+0000", 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8422000000000001e-05 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214770+0000", 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000264725 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59518 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21266432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:57.918 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.69680389199999, 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074662104000000007, 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59518 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140269+0000", 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140270+0000", 2024-09-12T09:32:57.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140272+0000", 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140279+0000", 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140285+0000", 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9619999999999998e-06 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214815+0000", 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074529564000000006 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214865+0000", 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9801e-05 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214931+0000", 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6711000000000006e-05 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59520 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21364736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.69678075100001, 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074898494999999995, 2024-09-12T09:32:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59520 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140292+0000", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140293+0000", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-07 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140295+0000", 2024-09-12T09:32:57.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140302+0000", 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140310+0000", 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3529999999999995e-06 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.214981+0000", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074671497000000003 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215079+0000", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7701999999999994e-05 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215191+0000", 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111697 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:59522 3.1d 3:bcb75868:::rbd_data.119172280d2a.0000000000000005:c [read 21463040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 164.696755792, 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.075032576000000004, 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 59522 2024-09-12T09:32:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140317+0000", 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140318+0000", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140320+0000", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140326+0000", 2024-09-12T09:32:57.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3840000000000002e-06 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.140331+0000", 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1309999999999997e-06 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215228+0000", 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.074896840000000006 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215279+0000", 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:13.215350+0000", 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1181999999999997e-05 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:116929 3.16 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 154.65340427800001, 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.12069165, 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116929 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183669+0000", 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.183670+0000", 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.421e-06 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211211+0000", 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027540630999999999 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211240+0000", 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211252+0000", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.202e-05 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211329+0000", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6694999999999997e-05 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.211446+0000", 2024-09-12T09:32:57.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011740500000000001 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.212488+0000", 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001041415 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.244656+0000", 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032168664 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304280+0000", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.059623509999999998 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304322+0000", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1656000000000002e-05 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.304360+0000", 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8858e-05 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117170 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 154.036926236, 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156297299999999, 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117170 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800147+0000", 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800149+0000", 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800158+0000", 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6090000000000001e-06 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800179+0000", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0945999999999999e-05 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800189+0000", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7969999999999995e-06 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.803990+0000", 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038006680000000001 2024-09-12T09:32:57.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804002+0000", 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2123000000000001e-05 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815766+0000", 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011764243 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815777+0000", 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0689999999999999e-05 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:117171 3.e 3.d28f38e (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 154.03682694599999, 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0156034199999999, 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:57.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117171 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800246+0000", 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800247+0000", 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1000000000000001e-06 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:57.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800257+0000", 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9380000000000004e-06 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800269+0000", 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1378e-05 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.800278+0000", 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804188+0000", 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039103439999999996 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:23.804197+0000", 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815839+0000", 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011641931 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:30:24.815850+0000", 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0559e-05 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:57.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:57.950 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:57.951 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:57.947+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:57.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:32:58.052 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:58.051+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:58.143 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:58.143 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:58.143 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:58.144 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:58.153 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:58.151+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:58.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:32:58.257 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:58.253+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:58.281 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:58.282 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:58.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:32:58.383 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:58.381+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:58.461 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:58.457+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708328215999998, 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:32:58.464 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:32:58.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708277402999997, 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:32:58.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708240739000004, 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.474 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:32:58.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708181405999994, 2024-09-12T09:32:58.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:32:58.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.483 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708151397999998, 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:32:58.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:32:58.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708132563000007, 2024-09-12T09:32:58.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:58.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:32:58.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708105845000006, 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.493 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:32:58.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708085079, 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:32:58.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:32:58.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708065364000007, 2024-09-12T09:32:58.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:32:58.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708035167999995, 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:32:58.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:32:58.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.708004599999995, 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:32:58.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:32:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707985797000006, 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:32:58.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707936703000001, 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.521 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:58.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:32:58.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707902207999993, 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:32:58.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.530 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707869430000002, 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:32:58.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707834298999998, 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.535 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:32:58.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707791975000006, 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:32:58.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707756467999999, 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:32:58.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:32:58.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707722197999999, 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:32:58.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:32:58.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.707668861000002, 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:32:58.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:32:58.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:58.559 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:58.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:32:58.562 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:58.559+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:58.665 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:32:58.661+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:32:58.730 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:58.730 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:58.730 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:32:58.730 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:58.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:32:58.766 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:58.767+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:58.867 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:32:58.861+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:32:58.920 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:58.920 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:32:58.920 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:32:58.920 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:32:58.920 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:58.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:32:58.967 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:58.967+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:59.069 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:32:59.067+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:32:59.102 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:32:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:32:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.886305198, 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:32:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.88418158, 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:32:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.884161753, 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:32:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:32:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:32:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:32:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:32:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:32:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.88206672, 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:32:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.880032113, 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:32:59.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.879993396, 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:32:59.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:32:59.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:32:59.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.879956412, 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:59.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.879942028, 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:32:59.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:32:59.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:32:59.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:32:59.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:59.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.879921037, 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.159 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:32:59.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:32:59.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.87990956599999, 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:32:59.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.879884081, 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:59.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:32:59.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:32:59.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.877696801, 2024-09-12T09:32:59.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:32:59.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:59.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.87768643299999, 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:32:59.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:32:59.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.877660467, 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:32:59.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:32:59.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.875968645, 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:32:59.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:32:59.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.875954089, 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:32:59.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.875933719, 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:32:59.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.875895091, 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:59.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:32:59.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.875880568, 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:32:59.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:32:59.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.87587015299999, 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:32:59.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:59.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:32:59.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:32:59.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:32:59.229 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:32:59.230 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:59.225+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:59.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:32:59.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:32:59.355 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:32:59.355+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:32:59.368 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:32:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:32:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:32:59.368 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:32:59.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:32:59.431 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:59.429+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:59.437 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:32:59.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:32:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:32:59.473 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:32:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:32:59.490 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:32:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:32:59.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:32:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:32:59.533 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:32:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:32:59.533 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:59.531+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:59.549 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:59.550 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:59.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:32:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:32:59.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:32:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:32:59.575 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:32:59.578 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:32:59.578 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:32:59.634 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:59.631+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:59.699 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:59.699 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:59.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:32:59.755 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:32:59.755+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:32:59.835 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:32:59.833+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:32:59.889 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:32:59.889 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:32:59.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:32:59.937 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:32:59.933+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:32:59.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-12T09:33:00.038 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:00.033+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:00.038 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.038 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:33:00.138 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:00.134+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:00.189 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.189 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:33:00.239 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:00.237+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:00.340 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:00.337+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:00.368 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.368 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:33:00.441 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:00.437+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:00.508 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.508 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:33:00.571 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:00.570+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:00.643 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:00.642+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:00.650 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.650 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:33:00.744 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:00.742+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:00.801 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.802 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.815 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:33:00.826 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:33:00.827 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:33:00.827 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-12T09:33:00.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-12T09:33:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-12T09:33:00.844 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:00.842+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:00.946 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:00.941+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:00.955 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:00.955 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:00.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:33:01.046 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:01.042+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:01.143 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:01.144 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:01.147 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:01.145+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:01.158 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:33:01.248 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:01.245+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:01.317 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:01.317 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:01.331 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:33:01.350 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:01.346+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:01.450 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:01.445+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:01.500 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:01.500 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:01.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:33:01.556 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:01.553+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:01.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:33:01.654 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:01.649+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:01.679 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:01.679 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:01.693 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:33:01.721 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:33:01.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:33:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:33:01.757 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:33:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:33:01.775 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:33:01.776 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:33:01.779 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:01.777+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:01.794 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:33:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:33:01.813 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:33:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:33:01.832 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:33:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:33:01.843 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-12T09:33:01.838+0000 7f345765a640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-12T09:33:01.845 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.001 2024-09-12T09:33:01.856 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:33:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:33:01.861 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:01.853+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:01.863 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:33:01.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:33:01.873 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:01.873 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:01.873 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:33:01.873 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:01.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:33:01.975 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:01.973+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:02.057 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:02.053+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:02.061 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:02.061 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:02.061 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:33:02.061 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:33:02.061 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:02.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:33:02.163 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:02.161+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:02.247 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189801 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490612708, 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047023563999999997, 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189801 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755274+0000", 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755275+0000", 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2999999999999999e-07 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755279+0000", 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0579999999999999e-06 2024-09-12T09:33:02.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755291+0000", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1572e-05 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755299+0000", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802129+0000", 2024-09-12T09:33:02.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046829825999999998 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802213+0000", 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4142999999999997e-05 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802298+0000", 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4967000000000002e-05 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189802 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1081344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490561894999999, 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047152594999999999, 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.253 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189802 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755325+0000", 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755326+0000", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755330+0000", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239999999999997e-06 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755341+0000", 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0879999999999999e-05 2024-09-12T09:33:02.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755350+0000", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1540000000000008e-06 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802334+0000", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046983694999999999 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802389+0000", 2024-09-12T09:33:02.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5099000000000002e-05 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802478+0000", 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8716999999999998e-05 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189803 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:33:02.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490525231000007, 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047356149, 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189803 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:33:02.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755362+0000", 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7199999999999998e-07 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755366+0000", 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4079999999999998e-06 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755375+0000", 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7259999999999997e-06 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755385+0000", 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4730000000000006e-06 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802523+0000", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047138342 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802607+0000", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000005e-05 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802718+0000", 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000110578 2024-09-12T09:33:02.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189804 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1114112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490465897999997, 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047472963, 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189804 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755421+0000", 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755422+0000", 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5799999999999999e-07 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755431+0000", 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2739999999999996e-06 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755438+0000", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-06 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755443+0000", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1580000000000004e-06 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802758+0000", 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047315299999999998 2024-09-12T09:33:02.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802823+0000", 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4523999999999994e-05 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802894+0000", 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0889000000000005e-05 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189806 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490435890000001, 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047690367999999997, 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189806 2024-09-12T09:33:02.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755451+0000", 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755452+0000", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8599999999999998e-07 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755454+0000", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755459+0000", 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755464+0000", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6460000000000001e-06 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.802940+0000", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047475971999999998 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803025+0000", 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4944000000000005e-05 2024-09-12T09:33:02.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803141+0000", 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116525 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189807 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490417054999995, 2024-09-12T09:33:02.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047844847000000003, 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189807 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755470+0000", 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755472+0000", 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755478+0000", 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5439999999999998e-06 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755490+0000", 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803178+0000", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047688214999999999 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803243+0000", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5135000000000006e-05 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803315+0000", 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1303000000000005e-05 2024-09-12T09:33:02.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189808 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490390336999994, 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048050692999999998, 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189808 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755496+0000", 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755497+0000", 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755499+0000", 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755507+0000", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9820000000000008e-06 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755511+0000", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3760000000000001e-06 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803361+0000", 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047849760999999998 2024-09-12T09:33:02.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803444+0000", 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240999999999998e-05 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803547+0000", 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102897 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189809 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1736704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490369571000002, 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048211038999999997, 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189809 2024-09-12T09:33:02.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755517+0000", 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755518+0000", 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755520+0000", 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8139999999999999e-06 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755525+0000", 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755530+0000", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803584+0000", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048054413999999997 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803647+0000", 2024-09-12T09:33:02.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2534999999999997e-05 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803728+0000", 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1586000000000001e-05 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189811 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1769472~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:33:02.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490349855999995, 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048432225000000002, 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189811 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755537+0000", 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755539+0000", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.796e-06 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755549+0000", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4420000000000004e-06 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755556+0000", 2024-09-12T09:33:02.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4660000000000002e-06 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803774+0000", 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048218365999999999 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803861+0000", 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6728e-05 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.803969+0000", 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000107963 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189812 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490319659999997, 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048574340000000001, 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189812 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755567+0000", 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755568+0000", 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755571+0000", 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.416e-06 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755579+0000", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.374e-06 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755589+0000", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804008+0000", 2024-09-12T09:33:02.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048418416999999998 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804068+0000", 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9959e-05 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804141+0000", 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3763999999999994e-05 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189814 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490289091999998, 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048759291000000003, 2024-09-12T09:33:02.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189814 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755598+0000", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755600+0000", 2024-09-12T09:33:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755606+0000", 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8259999999999998e-06 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755610+0000", 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.313e-06 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804186+0000", 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048575419000000002 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804272+0000", 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6408000000000003e-05 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804357+0000", 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4766999999999997e-05 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189816 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1212416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490270288999994, 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048900633999999998, 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189816 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755617+0000", 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755619+0000", 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9460000000000001e-06 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755626+0000", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0439999999999998e-06 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755637+0000", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0654e-05 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804393+0000", 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048756409000000001 2024-09-12T09:33:02.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804448+0000", 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5019000000000003e-05 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804517+0000", 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9134000000000006e-05 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189817 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1245184~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490221195000004, 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049092934999999997, 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189817 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755666+0000", 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7400000000000002e-07 2024-09-12T09:33:02.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755670+0000", 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3280000000000002e-06 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755680+0000", 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.01e-05 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755688+0000", 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3669999999999997e-06 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804563+0000", 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048874704999999997 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804650+0000", 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6833e-05 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804759+0000", 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010872800000000001 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189818 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490186699999995, 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049237191999999999, 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189818 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755700+0000", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755701+0000", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1200000000000002e-07 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755704+0000", 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0199999999999999e-06 2024-09-12T09:33:02.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755714+0000", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0504e-05 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755722+0000", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804805+0000", 2024-09-12T09:33:02.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049083006999999998 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804867+0000", 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1122000000000002e-05 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804937+0000", 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0722999999999994e-05 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189820 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1540096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490153922000005, 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049451892999999997, 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189820 2024-09-12T09:33:02.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755733+0000", 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755734+0000", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0200000000000001e-07 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755737+0000", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4539999999999999e-06 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755748+0000", 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0528e-05 2024-09-12T09:33:02.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755757+0000", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5060000000000001e-06 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.804984+0000", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049226598000000003 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805071+0000", 2024-09-12T09:33:02.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7499999999999999e-05 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805185+0000", 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011350500000000001 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189821 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1572864~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490118791, 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049597079000000002, 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189821 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755768+0000", 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755769+0000", 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2399999999999997e-07 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755780+0000", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1175999999999999e-05 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755790+0000", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0308e-05 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755799+0000", 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3720000000000005e-06 2024-09-12T09:33:02.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805221+0000", 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049422480999999997 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805280+0000", 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9132000000000002e-05 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805365+0000", 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4886e-05 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189822 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490076466999994, 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049801142999999999, 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189822 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755810+0000", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755811+0000", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5799999999999998e-07 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755814+0000", 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755825+0000", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0572e-05 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755833+0000", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2900000000000002e-06 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805410+0000", 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049577215000000001 2024-09-12T09:33:02.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805499+0000", 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8710999999999998e-05 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805611+0000", 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112339 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189824 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490040960000002, 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049917098999999999, 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189824 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755846+0000", 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755847+0000", 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7400000000000002e-07 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755850+0000", 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9560000000000002e-06 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755859+0000", 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8679999999999994e-06 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755869+0000", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3340000000000008e-06 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805649+0000", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049780416000000001 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805709+0000", 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9338e-05 2024-09-12T09:33:02.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805763+0000", 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4413000000000002e-05 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189825 3.10 3:0c2eaada:::rbd_data.1191c1f77e6b.0000000000000000:31 [read 1867776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.490006690000001, 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050120431999999999, 2024-09-12T09:33:02.333 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189825 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755880+0000", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755881+0000", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.92e-07 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755884+0000", 2024-09-12T09:33:02.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1839999999999999e-06 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755895+0000", 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0441999999999999e-05 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755903+0000", 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0679999999999997e-06 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805808+0000", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049905755000000003 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.805892+0000", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4061999999999996e-05 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806001+0000", 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010812899999999999 2024-09-12T09:33:02.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:189826 3.10 3:0f063a42:::rbd_data.1191c1f77e6b.0000000000000002:31 [read 1949696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.489953353000004, 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050220743999999998, 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 189826 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755933+0000", 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755934+0000", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8400000000000003e-07 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755938+0000", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000001e-06 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755948+0000", 2024-09-12T09:33:02.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0139999999999999e-05 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.755957+0000", 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8219999999999996e-06 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806037+0000", 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050079609999999997 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806098+0000", 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1511000000000001e-05 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:49.806154+0000", 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5906999999999999e-05 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.342 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:02.344 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:33:02.344 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:02.341+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:02.444 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:02.442+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:02.518 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:33:02.518 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:02.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:33:02.580 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:02.577+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:02.645 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:02.641+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:02.650 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:02.650 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:02.650 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:33:02.650 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:33:02.650 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:02.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:33:02.746 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:02.746+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:02.832 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.617002251, 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:33:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.614878633, 2024-09-12T09:33:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:33:02.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.614858806, 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:33:02.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.612763773, 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.853 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610729166, 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:33:02.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:33:02.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:33:02.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610690449, 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:33:02.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:33:02.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:02.864 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610653465, 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:33:02.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.869 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610639081, 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.61061809, 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:33:02.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:33:02.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610606619, 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:33:02.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:33:02.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.610581134, 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:33:02.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:02.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.608393854, 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:33:02.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:33:02.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.60838348599999, 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:33:02.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:33:02.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.60835752, 2024-09-12T09:33:02.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:33:02.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.606665698, 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:33:02.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:33:02.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.606651142, 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:02.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.606630772, 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:33:02.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:33:02.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.606592144, 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:33:02.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.606577621, 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:33:02.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.60656720599999, 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:33:02.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:02.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:02.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:33:02.937 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:02.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:02.938 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:02.938 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-12T09:33:02.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-12T09:33:02.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:33:02.941 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:02.937+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:02.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:33:03.043 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:03.037+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:03.077 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:03.077 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:03.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:33:03.147 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:03.146+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:03.217 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:33:03.217 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:33:03.217 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:03.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:33:03.275 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:03.274+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:03.342 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:03.341+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:03.397 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:03.397 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:33:03.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 160.19647140399999, 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:33:03.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 160.181500675, 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:33:03.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86465367100001, 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:33:03.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:33:03.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.864607312, 2024-09-12T09:33:03.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:33:03.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:33:03.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86450009199999, 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86440660100001, 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:33:03.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:33:03.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86434008800001, 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86427927899999, 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:33:03.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:33:03.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86421998700001, 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:03.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.864159478, 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:33:03.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86408142299999, 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:33:03.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:33:03.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86399604799999, 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:33:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86392772900001, 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:33:03.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:33:03.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86383514400001, 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86377923500001, 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:33:03.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:33:03.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86374691200001, 2024-09-12T09:33:03.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:33:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.863713319, 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:33:03.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.863662377, 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:33:03.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86356422200001, 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:33:03.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:03.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 150.86350712800001, 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:03.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:03.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:03.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:03.491 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:03.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:33:03.493 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:03.490+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:03.593 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:03.590+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:03.666 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:03.666 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:03.666 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:03.666 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:03.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:33:03.727 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:03.726+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:03.795 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:03.793+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:03.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:33:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:33:03.855 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:03.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:33:03.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:33:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.95 2024-09-12T09:33:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:33:03.915 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:03.914+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:03.997 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:03.994+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:33:04.015 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 161.14101306000001, 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 160.79348317200001, 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:04.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:33:04.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:33:04.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 160.791029262, 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:33:04.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 160.78891937500001, 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 132.29317273000001, 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:33:04.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:33:04.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065398213999998, 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065337651999997, 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:33:04.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065273375000004, 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:33:04.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:33:04.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065205856000006, 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:33:04.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065101229999996, 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:33:04.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:33:04.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:33:04.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.065037697999998, 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:04.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064999757999999, 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:33:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064971237999998, 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064921734999999, 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:33:04.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:33:04.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064877346000003, 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:33:04.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064815463000002, 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:33:04.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:33:04.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064767950000004, 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:33:04.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064739680000002, 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:33:04.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064712127000007, 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:33:04.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:33:04.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.064681343999993, 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:33:04.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:33:04.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:04.115 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:04.116 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:33:04.116 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:33:04.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:33:04.118 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:33:04.118 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:33:04.118 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:33:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:33:04.119 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:33:04.119 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:33:04.119 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:33:04.120 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:33:04.120 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:04.113+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:04.126 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:33:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:33:04.147 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:33:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:33:04.150 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:33:04.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:33:04.218 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:04.217+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:04.246 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:04.246 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:04.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:33:04.318 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:04.314+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:04.381 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:04.381 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:04.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:33:04.419 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:04.417+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:04.521 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:04.521+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:04.567 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:04.567 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:04.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:33:04.622 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:04.618+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:04.712 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:04.713 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:04.722 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:04.717+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:04.728 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:33:04.823 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:04.821+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:04.903 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:04.903 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:04.924 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:04.921+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:04.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:33:05.025 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:05.021+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:05.104 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:05.104 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:05.122 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:33:05.163 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:05.161+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:05.227 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:05.221+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:05.281 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:05.281 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:05.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:33:05.327 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:05.326+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:05.428 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:05.425+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:05.462 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:05.462 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:05.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:33:05.563 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:05.561+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:05.630 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:05.630+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:05.645 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:05.645 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:05.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:33:05.747 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:05.745+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:05.820 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:05.820 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:05.831 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:05.829+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:05.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:33:05.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:33:05.963 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:05.962+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:06.010 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:06.011 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:06.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:33:06.033 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:06.030+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:06.134 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:06.129+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:06.150 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:06.150 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:06.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-12T09:33:06.164 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:33:06.251 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:06.249+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:06.282 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:33:06.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:33:06.301 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:33:06.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:33:06.317 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:33:06.327 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:06.327 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:06.335 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:33:06.335 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:33:06.335 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:06.334+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:06.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:33:06.355 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:33:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:33:06.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:33:06.373 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:33:06.391 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:33:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:33:06.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:33:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:33:06.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:33:06.415 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:33:06.435 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:06.433+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:06.505 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:06.505 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:06.505 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-12T09:33:06.505 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-12T09:33:06.505 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:06.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:33:06.537 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:06.534+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:06.637 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:06.634+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171393 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.473291727, 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339148970000001, 2024-09-12T09:33:06.689 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171393 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213637+0000", 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213638+0000", 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213646+0000", 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213650+0000", 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213988+0000", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033728799999999999 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.213993+0000", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.463e-06 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247262+0000", 2024-09-12T09:33:06.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033269167 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247419+0000", 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015620599999999999 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.247552+0000", 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013294899999999999 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171408 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14565376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.471168109, 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033641818, 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171408 2024-09-12T09:33:06.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215760+0000", 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215761+0000", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215763+0000", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215769+0000", 2024-09-12T09:33:06.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4230000000000002e-06 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215774+0000", 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7729999999999999e-06 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215866+0000", 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2039000000000004e-05 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215870+0000", 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.54e-06 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249292+0000", 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334228889999999 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249350+0000", 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7670000000000002e-05 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249402+0000", 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1987000000000002e-05 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171409 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14630912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.471148282, 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033754249, 2024-09-12T09:33:06.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171409 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:06.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215780+0000", 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8299999999999998e-07 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215782+0000", 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3340000000000001e-06 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215785+0000", 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215788+0000", 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-09-12T09:33:06.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215886+0000", 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8178999999999996e-05 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.215890+0000", 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6179999999999999e-06 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249423+0000", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335332720000001 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249477+0000", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4391999999999997e-05 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.249534+0000", 2024-09-12T09:33:06.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6888999999999997e-05 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171425 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14647296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.469053249, 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033736032, 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171425 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217875+0000", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217877+0000", 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2219999999999999e-06 2024-09-12T09:33:06.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217884+0000", 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8180000000000004e-06 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.217887+0000", 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7379999999999999e-06 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218199+0000", 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00031172399999999999 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.218209+0000", 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0111e-05 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251509+0000", 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0332998090000001 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251559+0000", 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0232999999999998e-05 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.251611+0000", 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159e-05 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171435 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.467018642, 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033659608, 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171435 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219910+0000", 2024-09-12T09:33:06.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999996e-07 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219912+0000", 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219920+0000", 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219925+0000", 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220029+0000", 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103404 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1779999999999997e-06 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253486+0000", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03344959 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253523+0000", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7527000000000002e-05 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253569+0000", 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269e-05 2024-09-12T09:33:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171436 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14336000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.466979925, 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033741118, 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171436 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219948+0000", 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219949+0000", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219951+0000", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2290000000000002e-06 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219958+0000", 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1559999999999998e-06 2024-09-12T09:33:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219963+0000", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220076+0000", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011278299999999999 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220083+0000", 2024-09-12T09:33:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8419999999999999e-06 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253598+0000", 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335144119999999 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253646+0000", 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8162999999999999e-05 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253690+0000", 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3875999999999998e-05 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171437 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14401536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.466942941, 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033808426, 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171437 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219985+0000", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219986+0000", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219987+0000", 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-12T09:33:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219992+0000", 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.219995+0000", 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9969999999999999e-06 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220120+0000", 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012477699999999999 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220129+0000", 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7240000000000004e-06 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253717+0000", 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033587622 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253754+0000", 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6681000000000003e-05 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253794+0000", 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0330000000000002e-05 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171438 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.466928557, 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338942470000001, 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171438 2024-09-12T09:33:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220000+0000", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220001+0000", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.147e-06 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220004+0000", 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1889999999999999e-06 2024-09-12T09:33:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220007+0000", 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6019999999999998e-06 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220180+0000", 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017305399999999999 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220190+0000", 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0254999999999999e-05 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253819+0000", 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336281920000001 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253854+0000", 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5083000000000003e-05 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253894+0000", 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0445000000000003e-05 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171439 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14532608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.466907566, 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340052390000001, 2024-09-12T09:33:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171439 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220021+0000", 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4499999999999998e-07 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220022+0000", 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1030000000000001e-06 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220026+0000", 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3759999999999999e-06 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220028+0000", 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6170000000000001e-06 2024-09-12T09:33:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220222+0000", 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019390100000000001 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220226+0000", 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1710000000000004e-06 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253920+0000", 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0336941550000001 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.253956+0000", 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5654000000000001e-05 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254026+0000", 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0017000000000004e-05 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171440 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14598144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.466896095, 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341669499999999, 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.739 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171440 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220032+0000", 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220033+0000", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7699999999999992e-07 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220036+0000", 2024-09-12T09:33:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220039+0000", 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220271+0000", 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023173700000000001 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220282+0000", 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1178e-05 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254064+0000", 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03378255 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254134+0000", 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9398000000000002e-05 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254199+0000", 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5510000000000001e-05 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171441 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.46687061, 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0343133950000001, 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171441 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220058+0000", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4299999999999999e-07 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220059+0000", 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3039999999999999e-06 2024-09-12T09:33:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220068+0000", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4349999999999997e-06 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220073+0000", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220298+0000", 2024-09-12T09:33:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000224754 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.220301+0000", 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-06 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254236+0000", 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033934659 2024-09-12T09:33:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254305+0000", 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9233000000000006e-05 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.254371+0000", 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5888000000000003e-05 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171455 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15532032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.46468333, 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033743004, 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171455 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222245+0000", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222246+0000", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222249+0000", 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0130000000000002e-06 2024-09-12T09:33:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222252+0000", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5110000000000002e-06 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222513+0000", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026125200000000001 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222517+0000", 2024-09-12T09:33:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.676e-06 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255886+0000", 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333686959999999 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255926+0000", 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0408999999999999e-05 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.255988+0000", 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2096999999999999e-05 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171456 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.46467296199999, 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338395380000001, 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171456 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222255+0000", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222256+0000", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222257+0000", 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.333e-06 2024-09-12T09:33:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222263+0000", 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8300000000000001e-06 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222266+0000", 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6839999999999998e-06 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222550+0000", 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028362 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222559+0000", 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9779999999999994e-06 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256016+0000", 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0334568079999999 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256056+0000", 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9786999999999997e-05 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256095+0000", 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9266e-05 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171457 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.464646996, 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0339129250000001, 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171457 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222281+0000", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222282+0000", 2024-09-12T09:33:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222284+0000", 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222290+0000", 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222295+0000", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222574+0000", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027862399999999999 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.222578+0000", 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.619e-06 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256121+0000", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0335429060000001 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256156+0000", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5076000000000001e-05 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.256194+0000", 2024-09-12T09:33:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8631999999999999e-05 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171468 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15351808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.462955174, 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03374451, 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171468 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223973+0000", 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5499999999999999e-07 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223975+0000", 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.164e-06 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223978+0000", 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5669999999999999e-06 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223983+0000", 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6059999999999996e-06 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224202+0000", 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021930300000000001 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224206+0000", 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257565+0000", 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0333582910000001 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257638+0000", 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3832999999999996e-05 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257718+0000", 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9258000000000006e-05 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171469 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15417344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.462940618, 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03389927, 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171469 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223988+0000", 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223990+0000", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.223996+0000", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5809999999999996e-06 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224000+0000", 2024-09-12T09:33:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6829999999999999e-06 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224226+0000", 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022580000000000001 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224230+0000", 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257747+0000", 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033517317 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257819+0000", 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1987e-05 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257887+0000", 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7934000000000004e-05 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171470 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15482880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.462920248, 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0340590489999999, 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171470 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224008+0000", 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224009+0000", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224010+0000", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.807e-06 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224017+0000", 2024-09-12T09:33:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.776e-06 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224022+0000", 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6330000000000004e-06 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224249+0000", 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022730600000000001 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224253+0000", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9249999999999997e-06 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257916+0000", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.03366354 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.257989+0000", 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2496000000000005e-05 2024-09-12T09:33:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258067+0000", 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8190999999999997e-05 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171471 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15548416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.46288162, 2024-09-12T09:33:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034195559, 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171471 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224047+0000", 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224048+0000", 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3e-06 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224054+0000", 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9009999999999999e-06 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224057+0000", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224272+0000", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021455399999999999 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224275+0000", 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5530000000000001e-06 2024-09-12T09:33:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258098+0000", 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0338224140000001 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258174+0000", 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6255999999999996e-05 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258242+0000", 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8541000000000007e-05 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171472 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15613952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.462867097, 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034368261, 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171472 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224061+0000", 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224062+0000", 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000002e-07 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224063+0000", 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1179999999999999e-06 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224066+0000", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1259999999999998e-06 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224068+0000", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224294+0000", 2024-09-12T09:33:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022610299999999999 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224298+0000", 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258268+0000", 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.033969677 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258356+0000", 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7881999999999996e-05 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258430+0000", 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3633000000000005e-05 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4497.0:171473 3.12 3:497f7aa5:::rbd_data.1191e0d4a936.0000000000000002:25 [read 15679488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104)", 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 115.46285668199999, 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034540808, 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4497", 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 171473 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-12T09:33:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224072+0000", 2024-09-12T09:33:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224077+0000", 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224082+0000", 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029999999999997e-06 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-12T09:33:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224086+0000", 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224311+0000", 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022467900000000001 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:11.224315+0000", 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6890000000000002e-06 2024-09-12T09:33:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258461+0000", 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0341461409999999 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258527+0000", 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6357000000000002e-05 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-12T09:33:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-12T09:31:12.258613+0000", 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5356000000000001e-05 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-12T09:33:06.799 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:06.801 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:06.801+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:06.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-12T09:33:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:33:06.901 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:06.898+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:06.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-12T09:33:07.002 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:06.997+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:07.103 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:07.097+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:07.133 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:33:07.133 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:33:07.133 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:07.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-12T09:33:07.204 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:07.201+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:07.275 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:07.275 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:33:07.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:116934 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 22544384~7864320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.07461282200001, 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.10214039900000001, 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 116934 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198871+0000", 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.198873+0000", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3829999999999999e-06 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217904+0000", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019030399999999999 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217948+0000", 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4156000000000002e-05 2024-09-12T09:33:07.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.217965+0000", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7003e-05 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218063+0000", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7666999999999998e-05 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.218238+0000", 2024-09-12T09:33:07.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175472 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.219900+0000", 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016615449999999999 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.286844+0000", 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066944056000000002 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300893+0000", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049598 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300961+0000", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7414000000000002e-05 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301011+0000", 2024-09-12T09:33:07.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0705000000000003e-05 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116929 3.16 e63/51 3:6f24076f:::rbd_data.11916e3c78cf.0000000000000000:head v 63'158, mlcod=60'156)", 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.05964209300001, 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.089682022, 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213842+0000", 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.213844+0000", 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261780+0000", 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047935654000000001 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261802+0000", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1895e-05 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.261814+0000", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2326e-05 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262719+0000", 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00090449300000000003 2024-09-12T09:33:07.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.262747+0000", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8206999999999999e-05 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303491+0000", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.040743944999999997 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.303524+0000", 2024-09-12T09:33:07.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3047999999999999e-05 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122350 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742795089, 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051608323999999997, 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122350 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530689+0000", 2024-09-12T09:33:07.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530690+0000", 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8200000000000008e-07 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530694+0000", 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1470000000000003e-06 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530708+0000", 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4585e-05 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530719+0000", 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1005000000000001e-05 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.581985+0000", 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051265653000000001 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582060+0000", 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4775000000000002e-05 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582297+0000", 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023717700000000001 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122354 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74274872999999, 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051907253, 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122354 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530735+0000", 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530736+0000", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530741+0000", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3239999999999996e-06 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530756+0000", 2024-09-12T09:33:07.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5126999999999999e-05 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530767+0000", 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1515000000000001e-05 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582346+0000", 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051578378000000001 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582425+0000", 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9260999999999999e-05 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582642+0000", 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021752300000000001 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.296 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122355 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 868352~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74264151, 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052125642, 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122355 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.297 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530842+0000", 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530844+0000", 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530855+0000", 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0856000000000001e-05 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530883+0000", 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7552999999999999e-05 2024-09-12T09:33:07.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530902+0000", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9245999999999998e-05 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582685+0000", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.051782559999999998 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582762+0000", 2024-09-12T09:33:07.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7593000000000001e-05 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.582968+0000", 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020580699999999999 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122362 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:07.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742548019, 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052334526999999999, 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122362 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530936+0000", 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530937+0000", 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.364e-06 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530944+0000", 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530964+0000", 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0083000000000001e-05 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.530979+0000", 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5463000000000001e-05 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583018+0000", 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052038205999999997 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583076+0000", 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8464000000000001e-05 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583270+0000", 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019414399999999999 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122363 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74248150599999, 2024-09-12T09:33:07.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052564056999999997, 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122363 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:07.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531002+0000", 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531003+0000", 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531009+0000", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531028+0000", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8816000000000001e-05 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531043+0000", 2024-09-12T09:33:07.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4613000000000001e-05 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583304+0000", 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052261825999999997 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583375+0000", 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1018000000000003e-05 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583566+0000", 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019085099999999999 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122364 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742420697, 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052798291999999997, 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122364 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:07.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531063+0000", 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531064+0000", 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0359999999999999e-06 2024-09-12T09:33:07.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531070+0000", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8939999999999998e-06 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531089+0000", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729e-05 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531103+0000", 2024-09-12T09:33:07.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3973e-05 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583613+0000", 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052510243999999998 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583670+0000", 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7204000000000003e-05 2024-09-12T09:33:07.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583861+0000", 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019121200000000001 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122368 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742361405, 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194261999999999, 2024-09-12T09:33:07.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122368 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:07.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531122+0000", 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531123+0000", 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-06 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531129+0000", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531149+0000", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0662e-05 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531164+0000", 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4143e-05 2024-09-12T09:33:07.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.583907+0000", 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052743058000000002 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584013+0000", 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106866 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584317+0000", 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030320599999999998 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122369 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 786432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74230089599999, 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053598421, 2024-09-12T09:33:07.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122369 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531183+0000", 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531184+0000", 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.083e-06 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531203+0000", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9153000000000001e-05 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531222+0000", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9136000000000001e-05 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531236+0000", 2024-09-12T09:33:07.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4056999999999999e-05 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584378+0000", 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053141305 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584483+0000", 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010491 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584781+0000", 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029877700000000002 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122376 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 565248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742222841, 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053975691999999999, 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122376 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531261+0000", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531262+0000", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.353e-06 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531268+0000", 2024-09-12T09:33:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0739999999999998e-06 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531289+0000", 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0585999999999999e-05 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531321+0000", 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1668999999999998e-05 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584844+0000", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053523715999999999 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.584970+0000", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001261 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585237+0000", 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026619399999999998 2024-09-12T09:33:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122377 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742137466, 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054241064999999998, 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122377 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531346+0000", 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531347+0000", 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.094e-06 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531354+0000", 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2360000000000001e-06 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531374+0000", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9902999999999999e-05 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531393+0000", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9513000000000001e-05 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585295+0000", 2024-09-12T09:33:07.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053901484999999999 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585362+0000", 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7099000000000003e-05 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585587+0000", 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022573499999999999 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122378 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.742069147, 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054653417000000003, 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122378 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531415+0000", 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531416+0000", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.17e-06 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531422+0000", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7339999999999996e-06 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531463+0000", 2024-09-12T09:33:07.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1732e-05 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531480+0000", 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6946999999999999e-05 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585679+0000", 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054199012999999997 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.585804+0000", 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012461199999999999 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586068+0000", 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000264209 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122379 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74197656199999, 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054869091000000002, 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122379 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531507+0000", 2024-09-12T09:33:07.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531508+0000", 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0899999999999999e-06 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531515+0000", 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999996e-06 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531535+0000", 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0380000000000001e-05 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531550+0000", 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4489e-05 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586112+0000", 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054562787000000001 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586174+0000", 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1529e-05 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586376+0000", 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020231599999999999 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122382 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74192065299999, 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055232339999999998, 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122382 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531563+0000", 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531564+0000", 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1699999999999997e-07 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531568+0000", 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7330000000000001e-06 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531578+0000", 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0657e-05 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531585+0000", 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1029999999999996e-06 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586421+0000", 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054836048999999998 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586529+0000", 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107174 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586795+0000", 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026690699999999998 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122383 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74188832999999, 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055498100000000002, 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122383 2024-09-12T09:33:07.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531595+0000", 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531596+0000", 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7700000000000004e-07 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531599+0000", 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9459999999999998e-06 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531610+0000", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1032999999999999e-05 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531619+0000", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9630000000000004e-06 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586837+0000", 2024-09-12T09:33:07.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055217802000000003 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.586895+0000", 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7881999999999999e-05 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587094+0000", 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019879699999999999 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122384 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74185473700001, 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055873556999999997, 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122384 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531629+0000", 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531630+0000", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5600000000000005e-07 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531633+0000", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4340000000000001e-06 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531648+0000", 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4932999999999999e-05 2024-09-12T09:33:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531662+0000", 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3686000000000001e-05 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587141+0000", 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055479128000000003 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587252+0000", 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011084 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587503+0000", 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025087999999999998 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122391 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74180379500001, 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056148627999999999, 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122391 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531680+0000", 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531681+0000", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1570000000000001e-06 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531687+0000", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4260000000000002e-06 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531729+0000", 2024-09-12T09:33:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2803000000000003e-05 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531750+0000", 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0613999999999999e-05 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587560+0000", 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055809676000000003 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587627+0000", 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6840000000000004e-05 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587829+0000", 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020211200000000001 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122392 3.4 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74170563999999, 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056449324000000002, 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122392 2024-09-12T09:33:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531778+0000", 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531780+0000", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8169999999999999e-06 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531788+0000", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.627e-06 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531806+0000", 2024-09-12T09:33:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8362000000000001e-05 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531818+0000", 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1688000000000001e-05 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587874+0000", 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056056679999999998 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.587982+0000", 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107792 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588227+0000", 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024535799999999999 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:122397 3.4 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:14 [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 154.74164854599999, 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056748492999999997, 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122397 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531835+0000", 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531836+0000", 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.142e-06 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531841+0000", 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5979999999999999e-06 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531856+0000", 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4524e-05 2024-09-12T09:33:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.531866+0000", 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0558e-05 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588258+0000", 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.056392214000000003 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588340+0000", 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1302000000000002e-05 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:32.588584+0000", 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024415500000000002 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.375 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:07.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-12T09:33:07.377 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:07.373+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:07.478 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:07.477+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:07.509 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:07.509 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:07.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-12T09:33:07.579 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:07.574+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:07.644 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:33:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:33:07.644 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:07.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-12T09:33:07.680 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:07.677+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:07.780 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:07.778+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:112371 3.4 e63/23 3:2121a1c3:::rbd_data.11916e3c78cf.0000000000000006:head v 63'53, mlcod=59'51)", 2024-09-12T09:33:07.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.958449696, 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034464697000000002, 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874619+0000", 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.874622+0000", 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729999999999999e-06 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888799+0000", 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014176714 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888831+0000", 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2722999999999998e-05 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888847+0000", 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5262999999999999e-05 2024-09-12T09:33:07.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888940+0000", 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3572000000000005e-05 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.888972+0000", 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1482000000000001e-05 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909011+0000", 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020039251000000001 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:22.909084+0000", 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3219000000000005e-05 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116934 3.4 e63/23 3:21f3f29f:::rbd_data.11916e3c78cf.0000000000000003:head v 63'55, mlcod=63'53)", 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.61091980800001, 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.078528130000000002, 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:07.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222149+0000", 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.222152+0000", 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7949999999999999e-06 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237563+0000", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015410469 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237660+0000", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6982000000000001e-05 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237694+0000", 2024-09-12T09:33:07.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4118000000000001e-05 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237797+0000", 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010343100000000001 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.237826+0000", 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592999999999999e-05 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300567+0000", 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062741533000000002 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.300677+0000", 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116931 3.14 e63/54 3:2dda4407:::rbd_data.11916e3c78cf.0000000000000001:head v 63'450, mlcod=62'447)", 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.60846589799999, 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076564705999999996, 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224603+0000", 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.224606+0000", 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6589999999999999e-06 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270927+0000", 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046320676999999998 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270954+0000", 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7426000000000001e-05 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.270966+0000", 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1932e-05 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271047+0000", 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1024000000000002e-05 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.271077+0000", 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0409e-05 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301124+0000", 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030047002999999999 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.301168+0000", 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3575999999999997e-05 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4497.0:116937 3.1e e63/46 3:7880c4ca:::rbd_data.11916e3c78cf.0000000000000005:head v 63'103, mlcod=60'101)", 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 164.606356011, 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.071677749999999998, 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:07.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226713+0000", 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.226715+0000", 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0219999999999999e-06 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256332+0000", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.029616825999999999 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256359+0000", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7183999999999999e-05 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256373+0000", 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3389e-05 2024-09-12T09:33:07.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256733+0000", 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036068800000000001 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.256764+0000", 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1204999999999997e-05 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298343+0000", 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.041578581000000003 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:23.298391+0000", 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7855000000000002e-05 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.855 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:141950 3.4 3:23110369:::rbd_data.11913a2849d0.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 136.11060936600001, 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.050547628999999997, 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141950 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722460+0000", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722462+0000", 2024-09-12T09:33:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.356e-06 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722474+0000", 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1763000000000001e-05 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722507+0000", 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2907000000000002e-05 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722528+0000", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1586e-05 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722693+0000", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164191 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.722832+0000", 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013980500000000001 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.732968+0000", 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010135745 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772744+0000", 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.039776230000000003 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772842+0000", 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7641999999999998e-05 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.772934+0000", 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2081000000000001e-05 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:30:51.773007+0000", 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3323000000000003e-05 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177237 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88283484999999, 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031598996999999997, 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177237 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950234+0000", 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950235+0000", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950244+0000", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3839999999999997e-06 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950261+0000", 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6996000000000001e-05 2024-09-12T09:33:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950276+0000", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5075999999999999e-05 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981526+0000", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031249593999999999 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981667+0000", 2024-09-12T09:33:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014181900000000001 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981833+0000", 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000165935 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177238 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 8650752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88277428799999, 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031872662000000003, 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177238 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950295+0000", 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950296+0000", 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.540000000000001e-07 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950302+0000", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0159999999999997e-06 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950322+0000", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9843999999999998e-05 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950334+0000", 2024-09-12T09:33:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2736e-05 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.981909+0000", 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031574998 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982020+0000", 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011024899999999999 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982167+0000", 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014786500000000001 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177239 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 10223616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882710011, 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032193881000000001, 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177239 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950359+0000", 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950360+0000", 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950366+0000", 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950383+0000", 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7125999999999998e-05 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950405+0000", 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-05 2024-09-12T09:33:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982249+0000", 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.031844461999999997 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982381+0000", 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013217200000000001 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982553+0000", 2024-09-12T09:33:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017153099999999999 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177240 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882642492, 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032463135999999997, 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177240 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950427+0000", 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950428+0000", 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.72e-06 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950436+0000", 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7999999999999999e-06 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950453+0000", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6986999999999998e-05 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950465+0000", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2363e-05 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982630+0000", 2024-09-12T09:33:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032164737999999998 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982746+0000", 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115779 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982890+0000", 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014374900000000001 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177241 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88253786600001, 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032771522999999997, 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177241 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950531+0000", 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950533+0000", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.536e-06 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950539+0000", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1600000000000003e-06 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950564+0000", 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4624e-05 2024-09-12T09:33:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950577+0000", 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3406e-05 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.982994+0000", 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032416599999999997 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983119+0000", 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012550500000000001 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983303+0000", 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000183692 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177242 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 15990784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88247433399999, 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033041400999999998, 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177242 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950595+0000", 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950596+0000", 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950599+0000", 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2660000000000002e-06 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950616+0000", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6563e-05 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950623+0000", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.204e-06 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983377+0000", 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.032754276999999998 2024-09-12T09:33:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983490+0000", 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112809 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983636+0000", 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000146358 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177243 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 17563648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882436394, 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033415274000000002, 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177243 2024-09-12T09:33:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950633+0000", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950636+0000", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.864e-06 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950645+0000", 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8130000000000002e-06 2024-09-12T09:33:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950652+0000", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983717+0000", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033065196999999998 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.983843+0000", 2024-09-12T09:33:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126026 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984048+0000", 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020521100000000001 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177244 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 18874368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88240787399999, 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033773446999999998, 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177244 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950661+0000", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950662+0000", 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-12T09:33:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950665+0000", 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950676+0000", 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253999999999999e-05 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950688+0000", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2319e-05 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984155+0000", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033466424000000002 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984281+0000", 2024-09-12T09:33:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012666500000000001 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984435+0000", 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015345199999999999 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177245 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882358371, 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034118884000000002, 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177245 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950711+0000", 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950712+0000", 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950725+0000", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2982999999999999e-05 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950736+0000", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0937e-05 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950743+0000", 2024-09-12T09:33:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984537+0000", 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.033793714000000002 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984656+0000", 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011927200000000001 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984830+0000", 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173172 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177246 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 23330816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882313982, 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034445577999999998, 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177246 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950755+0000", 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950756+0000", 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6700000000000003e-07 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950759+0000", 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950772+0000", 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-05 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950800+0000", 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8385999999999999e-05 2024-09-12T09:33:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.984922+0000", 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034121314999999999 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985042+0000", 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120615 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985201+0000", 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015836499999999999 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177247 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 24903680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882252099, 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034839725000000002, 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177247 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950817+0000", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950818+0000", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.33e-07 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950824+0000", 2024-09-12T09:33:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6239999999999999e-06 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950843+0000", 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9896000000000001e-05 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950853+0000", 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6569999999999992e-06 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985292+0000", 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034438930999999999 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985413+0000", 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121419 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985657+0000", 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000243265 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177248 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 26214400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882204586, 2024-09-12T09:33:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035205320999999998, 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177248 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950865+0000", 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950868+0000", 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.92e-06 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950877+0000", 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5569999999999996e-06 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950883+0000", 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985749+0000", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.034866028 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.985894+0000", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014436200000000001 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986070+0000", 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017601800000000001 2024-09-12T09:33:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177249 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 27787264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882176316, 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035643466999999998, 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177249 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950893+0000", 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.13e-07 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950896+0000", 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6699999999999998e-06 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950904+0000", 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4039999999999995e-06 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950911+0000", 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6390000000000003e-06 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986176+0000", 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035264477000000002 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986310+0000", 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000134599 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986536+0000", 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022606500000000001 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177250 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 30670848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.882148763, 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036100843000000001, 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177250 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950920+0000", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950921+0000", 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-12T09:33:07.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950924+0000", 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6070000000000002e-06 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950932+0000", 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3629999999999994e-06 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950941+0000", 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7099999999999996e-06 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986651+0000", 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.035710565999999999 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.986802+0000", 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015128299999999999 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987021+0000", 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021871400000000001 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:177251 3.4 3:22ecac6c:::rbd_data.11914d78e5cf.0000000000000000:2a [sparse-read 32243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.88211798, 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036498383000000002, 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177251 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950951+0000", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950952+0000", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3e-07 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950954+0000", 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4830000000000002e-06 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950963+0000", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2800000000000003e-06 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.950969+0000", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4799999999999998e-06 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987067+0000", 2024-09-12T09:33:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.036098462999999997 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987217+0000", 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000150032 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.987450+0000", 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023201499999999999 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:07.935 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:07.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-12T09:33:07.937 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:07.934+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:08.037 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:08.033+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:08.118 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:08.118 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:08.118 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:08.118 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:08.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-12T09:33:08.138 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:08.133+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:08.239 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:08.233+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:08.261 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-12T09:33:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-12T09:33:08.261 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:08.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-12T09:33:08.339 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:08.338+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148932 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.96535615499999, 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097739540999999999, 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148932 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435684+0000", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435685+0000", 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1399999999999997e-07 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435691+0000", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435698+0000", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5590000000000002e-06 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435702+0000", 2024-09-12T09:33:08.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533123+0000", 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097420925000000005 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533195+0000", 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1876000000000001e-05 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533424+0000", 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000228812 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148934 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 958464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.965330924, 2024-09-12T09:33:08.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098072909999999999, 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148934 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435710+0000", 2024-09-12T09:33:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435712+0000", 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1150000000000001e-06 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435719+0000", 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5560000000000004e-06 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435724+0000", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0440000000000003e-06 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533461+0000", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097737245 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533542+0000", 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0659000000000001e-05 2024-09-12T09:33:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533782+0000", 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024089900000000001 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148937 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 968704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.965293004, 2024-09-12T09:33:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098484904999999998, 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148937 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435747+0000", 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435748+0000", 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435751+0000", 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9019999999999999e-06 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435759+0000", 2024-09-12T09:33:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.174e-06 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435767+0000", 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1580000000000005e-06 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533814+0000", 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098046878000000004 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.533884+0000", 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000005e-05 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534232+0000", 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034813299999999998 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:148939 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.965263003, 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098828796999999996, 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 148939 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435777+0000", 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435778+0000", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435781+0000", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435790+0000", 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9509999999999995e-06 2024-09-12T09:33:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.435795+0000", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6550000000000001e-06 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534267+0000", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471600000000006 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534343+0000", 2024-09-12T09:33:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6416999999999997e-05 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.534606+0000", 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026303699999999998 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149093 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 842752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:33:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.957228582, 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097672901000000006, 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149093 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:33:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443812+0000", 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443813+0000", 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4300000000000002e-07 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443816+0000", 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443828+0000", 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.24e-05 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.443835+0000", 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0099999999999998e-06 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541188+0000", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097352346000000006 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541260+0000", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2249999999999994e-05 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541485+0000", 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022496199999999999 2024-09-12T09:33:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149095 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95690770900001, 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097697956000000002, 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149095 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444133+0000", 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444134+0000", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1710000000000001e-06 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444139+0000", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1610000000000002e-06 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444151+0000", 2024-09-12T09:33:08.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2026e-05 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444159+0000", 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541517+0000", 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097357942000000003 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541595+0000", 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8059000000000006e-05 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.541831+0000", 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000235713 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149102 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 875520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.956432404, 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097924780000000003, 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149102 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444608+0000", 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444609+0000", 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444612+0000", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444620+0000", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2539999999999992e-06 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444626+0000", 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0390000000000001e-06 2024-09-12T09:33:08.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542213+0000", 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097586874000000004 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542301+0000", 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8088000000000001e-05 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542533+0000", 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000231747 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149103 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 885760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95640365600001, 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098232021000000003, 2024-09-12T09:33:08.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149103 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444637+0000", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444640+0000", 2024-09-12T09:33:08.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.118e-06 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444648+0000", 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.444654+0000", 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542566+0000", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097912664999999996 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542648+0000", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1236000000000006e-05 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542869+0000", 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022126700000000001 2024-09-12T09:33:08.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149105 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95585319, 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097985693999999998, 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149105 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445187+0000", 2024-09-12T09:33:08.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445188+0000", 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.035e-06 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445193+0000", 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445206+0000", 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3013999999999999e-05 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445214+0000", 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7389999999999999e-06 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542901+0000", 2024-09-12T09:33:08.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097687214999999994 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.542986+0000", 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4815999999999996e-05 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543173+0000", 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018672400000000001 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149107 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 906240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.955810368, 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098227351000000004, 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149107 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445230+0000", 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445231+0000", 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445234+0000", 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8940000000000002e-06 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445242+0000", 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1890000000000007e-06 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445264+0000", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1885000000000001e-05 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543206+0000", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097942277999999994 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543281+0000", 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5158000000000001e-05 2024-09-12T09:33:08.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543457+0000", 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017634899999999999 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149110 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.955764477, 2024-09-12T09:33:08.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098472535999999999, 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149110 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:33:08.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445276+0000", 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445277+0000", 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445280+0000", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445335+0000", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5318999999999997e-05 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445345+0000", 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0456e-05 2024-09-12T09:33:08.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543491+0000", 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098145783 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543575+0000", 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4155999999999999e-05 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543748+0000", 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000173139 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149118 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 918528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.955252919, 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098246002999999998, 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149118 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445788+0000", 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445789+0000", 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445794+0000", 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2809999999999999e-06 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445807+0000", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3566e-05 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445820+0000", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2456000000000001e-05 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543781+0000", 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.097960795000000003 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.543858+0000", 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7472000000000006e-05 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544034+0000", 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000175341 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149120 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 928768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95520514, 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098510501, 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149120 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445835+0000", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445836+0000", 2024-09-12T09:33:08.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2200000000000004e-07 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445839+0000", 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3529999999999999e-06 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445847+0000", 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445854+0000", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.421e-06 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544065+0000", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098211693000000003 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544151+0000", 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5632999999999998e-05 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544346+0000", 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194802 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149122 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 939008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.955149494, 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098758077999999999, 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149122 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445891+0000", 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445892+0000", 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3499999999999995e-07 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445896+0000", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8589999999999998e-06 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445904+0000", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8419999999999994e-06 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445910+0000", 2024-09-12T09:33:08.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0510000000000002e-06 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544382+0000", 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098471632000000003 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544460+0000", 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8419999999999998e-05 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544649+0000", 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018853899999999999 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149123 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 949248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.955117473, 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099124331999999996, 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149123 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445923+0000", 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445930+0000", 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7730000000000002e-06 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445938+0000", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4769999999999997e-06 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.445943+0000", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0660000000000003e-06 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544710+0000", 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098767355000000001 2024-09-12T09:33:08.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.544792+0000", 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2231000000000006e-05 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545047+0000", 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025494200000000003 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149126 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.954553324, 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098922344999999995, 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149126 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446487+0000", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446488+0000", 2024-09-12T09:33:08.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0920000000000001e-06 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446493+0000", 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0320000000000003e-06 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446508+0000", 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.501e-05 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446516+0000", 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8839999999999994e-06 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545092+0000", 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098575542000000002 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545145+0000", 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3669999999999999e-05 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545409+0000", 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026411499999999999 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149128 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 969728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95415942299999, 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098898016000000005, 2024-09-12T09:33:08.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149128 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446881+0000", 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446882+0000", 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.569999999999999e-07 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.446887+0000", 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.583e-06 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447247+0000", 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036053399999999997 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447258+0000", 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0560999999999999e-05 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545459+0000", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098200867999999997 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545533+0000", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4303999999999999e-05 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545779+0000", 2024-09-12T09:33:08.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000246209 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:149129 3.19 3:9db6df72:::rbd_data.1191e31dd75.0000000000000005:23 [read 979968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96)", 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.95376772500001, 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.099046217000000006, 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 149129 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447273+0000", 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7499999999999999e-07 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447278+0000", 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0459999999999999e-06 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447304+0000", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6511000000000001e-05 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.447312+0000", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750000000000005e-06 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545879+0000", 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.098566691999999997 2024-09-12T09:33:08.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.545968+0000", 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8718e-05 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:00.546319+0000", 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035129999999999997 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:176801 3.1e 3:7a481924:::rbd_header.1191f7cb0ecd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116)", 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.585901607, 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87643883, 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176801 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815139+0000", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815142+0000", 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2760000000000001e-06 2024-09-12T09:33:08.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815156+0000", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4226000000000001e-05 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815185+0000", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8478000000000001e-05 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815199+0000", 2024-09-12T09:33:08.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4384e-05 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815279+0000", 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0290000000000005e-05 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:24.815289+0000", 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2220000000000008e-06 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690425+0000", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87513584099999997 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.690717+0000", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029259700000000002 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:25.691578+0000", 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086051599999999997 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4497.0:190822 3.12 3:4c1ec2d7:::rbd_data.11918e3fc64.0000000000000001:head [stat,truncate 19660800] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.727424811000006, 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1257150199999999, 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4497", 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2567730644", 2024-09-12T09:33:08.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 190822 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673616+0000", 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673618+0000", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673627+0000", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8799999999999997e-06 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673647+0000", 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0530000000000002e-05 2024-09-12T09:33:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673662+0000", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.467e-05 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673765+0000", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102988 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.673782+0000", 2024-09-12T09:33:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7597000000000001e-05 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676952+0000", 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031693070000000001 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:52.676965+0000", 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3593e-05 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-12T09:33:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.786543+0000", 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1095780369999999 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787032+0000", 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000488989 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0,7", 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.787550+0000", 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051780900000000004 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.789977+0000", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024266679999999999 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.791509+0000", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001532177 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798783+0000", 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072740010000000004 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.798957+0000", 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017402300000000001 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-12T09:31:53.799331+0000", 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000373736 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-12T09:33:08.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-12T09:33:08.502 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:08.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-12T09:33:08.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-12T09:33:08.504 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:08.501+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:08.604 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-12T09:33:08.602+0000 7fea4f979640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26387) UID: 0 2024-09-12T09:33:08.637 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-12T09:33:08.637 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-12T09:33:08.637 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-12T09:33:08.637 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-12T09:33:08.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-12T09:33:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:33:08.700 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-12T09:33:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-12T09:33:08.705 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-12T09:33:08.702+0000 7f7418ae8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26390) UID: 0 2024-09-12T09:33:08.717 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-12T09:33:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-12T09:33:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-12T09:33:08.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-12T09:33:08.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-12T09:33:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-12T09:33:08.771 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-12T09:33:08.771 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-12T09:33:08.774 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:08.774 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:08.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-12T09:33:08.792 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-12T09:33:08.792 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-12T09:33:08.813 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-12T09:33:08.859 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:33:08.859 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-12T09:33:08.859 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-12T09:33:08.860 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:08.829+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:08.907 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:08.907 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:08.905+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:08.922 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-12T09:33:08.924 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 7, 6, 1, 0] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-12T09:33:08.924 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-12T09:33:08.925 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-12T09:33:08.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-12T09:33:08.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-12T09:33:09.019 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:09.018+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:09.097 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.098 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.109 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-09-12T09:33:09.105+0000 7f4324946640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26533) UID: 0 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-12T09:33:09.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.114 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-12T09:33:09.210 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:09.206+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:09.279 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.280 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.313 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-12T09:33:09.339 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:09.337+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:09.411 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:09.409+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:09.416 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.416 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-12T09:33:09.514 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.515 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.516 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-09-12T09:33:09.513+0000 7f2db2662640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26531) UID: 0 2024-09-12T09:33:09.518 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.518 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-12T09:33:09.593 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.593 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.608 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-12T09:33:09.651 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:09.649+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.674 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.677 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-12T09:33:09.714 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:09.709+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:09.771 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.771 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-12T09:33:09.827 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-09-12T09:33:09.825+0000 7f4f9d27e640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26535) UID: 0 2024-09-12T09:33:09.829 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.832 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-12T09:33:09.916 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-12T09:33:09.914+0000 7f58fef0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26389) UID: 0 2024-09-12T09:33:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-12T09:33:09.957 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:09.957 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:09.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:09.993 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:09.997 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:09.997 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:09.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-12T09:33:10.023 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-09-12T09:33:10.021+0000 7fdc23f35640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26537) UID: 0 2024-09-12T09:33:10.047 INFO:teuthology.orchestra.run.smithi007.stdout:All operations completed A-OK! 2024-09-12T09:33:10.085 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:10.086 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-12T09:33:10.095 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-12T09:33:10.142 INFO:tasks.thrashosds:joining thrashosds 2024-09-12T09:33:10.143 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:10.143 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:10.144 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-12T09:33:10.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:10.196 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:10.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-12T09:33:10.312 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:10.313 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:10.318 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-12T09:33:10.321 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-12T09:33:10.321 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-12T09:33:10.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-12T09:33:10.448 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-12T09:33:10.449 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-12T09:33:11.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-12T09:33:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-12T09:33:13.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-12T09:33:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-12T09:33:15.322 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-12T09:33:15.323 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-12T09:33:15.323 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T09:33:15.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T09:33:15.325 INFO:tasks.ceph.osd.0:Started 2024-09-12T09:33:15.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:33:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:config diff 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:config diff get 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:config get 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:config help [] 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:config set ... 2024-09-12T09:33:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:admin_socket: invalid command 2024-09-12T09:33:15.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:15.454 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-12T09:33:16.409 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:33:16.406+0000 7f2c97a527c0 -1 Falling back to public interface 2024-09-12T09:33:20.369 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:33:20.366+0000 7f2c97a527c0 -1 osd.0 140 log_to_monitors true 2024-09-12T09:33:20.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-12T09:33:20.579 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-12T09:33:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-12T09:33:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-12T09:33:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-12T09:33:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator fragmentation block 2024-09-12T09:33:20.580 INFO:teuthology.orchestra.run.smithi007.stderr:admin_socket: invalid command 2024-09-12T09:33:20.582 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-12T09:33:20.582 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-12T09:33:21.220 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-12T09:33:21.218+0000 7f2c8b82d640 -1 osd.0 215 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T09:33:25.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-12T09:33:25.743 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:25.743 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-12T09:33:25.743 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-12T09:33:25.743 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:25.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-12T09:33:25.913 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-12T09:33:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:33:25.927 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-12T09:33:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:33:25.936 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-12T09:33:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-12T09:33:25.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-12T09:33:26.727 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-12T09:33:26.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-12T09:33:27.723 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-12T09:33:27.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-12T09:33:28.751 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-12T09:33:28.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-12T09:33:29.747 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-12T09:33:29.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-12T09:33:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-12T09:33:30.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-12T09:33:31.768 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-12T09:33:31.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-12T09:33:32.776 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-12T09:33:32.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-12T09:33:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-12T09:33:33.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-12T09:33:34.796 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-12T09:33:34.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-12T09:33:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-12T09:33:35.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-12T09:33:36.809 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.6 to 1 (10000) 2024-09-12T09:33:36.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-12T09:33:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-12T09:33:37.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-12T09:33:38.698 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-12T09:33:38.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-12T09:33:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-12T09:33:39.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-12T09:33:40.850 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-12T09:33:40.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-12T09:33:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-12T09:33:41.880 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-12T09:33:41.880 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-12T09:33:41.880 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-12T09:33:41.880 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-12T09:33:41.881 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T09:33:41.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:33:42.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:42.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":236,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:33:41.851479+0000","last_up_change":"2024-09-12T09:33:22.142642+0000","last_in_change":"2024-09-12T09:31:51.831699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T09:29:22.198256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T09:29:32.744560+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":216,"up_thru":220,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6817","nonce":1479059197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6819","nonce":1479059197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6823","nonce":1479059197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6821","nonce":1479059197}]},"public_addr":"172.21.15.7:6817/1479059197","cluster_addr":"172.21.15.7:6819/1479059197","heartbeat_back_addr":"172.21.15.7:6823/1479059197","heartbeat_front_addr":"172.21.15.7:6821/1479059197","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":226,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[2,6]},{"pgid":"3.2","osds":[0,6]},{"pgid":"3.e","osds":[5,0]},{"pgid":"3.12","osds":[7,2]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:33:42.217 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T09:33:42.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:33:42.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:42.526 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":236,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:33:41.851479+0000","last_up_change":"2024-09-12T09:33:22.142642+0000","last_in_change":"2024-09-12T09:31:51.831699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T09:29:22.198256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T09:29:32.744560+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":216,"up_thru":220,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6817","nonce":1479059197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6819","nonce":1479059197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6823","nonce":1479059197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6821","nonce":1479059197}]},"public_addr":"172.21.15.7:6817/1479059197","cluster_addr":"172.21.15.7:6819/1479059197","heartbeat_back_addr":"172.21.15.7:6823/1479059197","heartbeat_front_addr":"172.21.15.7:6821/1479059197","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":226,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[2,6]},{"pgid":"3.2","osds":[0,6]},{"pgid":"3.e","osds":[5,0]},{"pgid":"3.12","osds":[7,2]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:33:42.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T09:33:42.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T09:33:42.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T09:33:42.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T09:33:42.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T09:33:42.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T09:33:42.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T09:33:42.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T09:33:42.814 INFO:teuthology.orchestra.run.smithi007.stdout:64424509498 2024-09-12T09:33:42.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T09:33:42.816 INFO:teuthology.orchestra.run.smithi007.stdout:64424509497 2024-09-12T09:33:42.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T09:33:42.823 INFO:teuthology.orchestra.run.smithi007.stdout:927712935942 2024-09-12T09:33:42.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T09:33:42.832 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:42.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T09:33:42.835 INFO:teuthology.orchestra.run.smithi007.stdout:60129542203 2024-09-12T09:33:42.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T09:33:42.838 INFO:teuthology.orchestra.run.smithi007.stdout:60129542201 2024-09-12T09:33:42.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T09:33:42.841 INFO:teuthology.orchestra.run.smithi007.stdout:60129542203 2024-09-12T09:33:42.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T09:33:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:42.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T09:33:43.362 INFO:teuthology.orchestra.run.smithi007.stdout:64424509495 2024-09-12T09:33:43.374 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:43.377 INFO:teuthology.orchestra.run.smithi007.stdout:64424509496 2024-09-12T09:33:43.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509495 for osd.7 2024-09-12T09:33:43.387 INFO:teuthology.orchestra.run.smithi007.stdout:927712935941 2024-09-12T09:33:43.396 INFO:teuthology.orchestra.run.smithi007.stdout:60129542200 2024-09-12T09:33:43.400 INFO:teuthology.orchestra.run.smithi007.stdout:60129542201 2024-09-12T09:33:43.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542202 for osd.2 2024-09-12T09:33:43.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509498 got 64424509496 for osd.4 2024-09-12T09:33:43.419 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:43.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 927712935942 got 927712935941 for osd.0 2024-09-12T09:33:43.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542201 got 60129542200 for osd.1 2024-09-12T09:33:43.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.6 2024-09-12T09:33:43.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542202 for osd.3 2024-09-12T09:33:43.459 INFO:teuthology.orchestra.run.smithi007.stdout:60129542201 2024-09-12T09:33:43.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542201 for osd.5 2024-09-12T09:33:44.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T09:33:44.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T09:33:44.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T09:33:44.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T09:33:44.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T09:33:44.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T09:33:44.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T09:33:44.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T09:33:44.949 INFO:teuthology.orchestra.run.smithi007.stdout:64424509497 2024-09-12T09:33:44.974 INFO:teuthology.orchestra.run.smithi007.stdout:927712935942 2024-09-12T09:33:44.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509497 for osd.7 2024-09-12T09:33:44.976 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:44.980 INFO:teuthology.orchestra.run.smithi007.stdout:60129542203 2024-09-12T09:33:45.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 927712935942 got 927712935942 for osd.0 2024-09-12T09:33:45.000 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542203 for osd.2 2024-09-12T09:33:45.002 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.003 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:45.010 INFO:teuthology.orchestra.run.smithi007.stdout:64424509498 2024-09-12T09:33:45.015 INFO:teuthology.orchestra.run.smithi007.stdout:60129542201 2024-09-12T09:33:45.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542202 for osd.6 2024-09-12T09:33:45.028 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509498 got 64424509498 for osd.4 2024-09-12T09:33:45.032 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.033 INFO:teuthology.orchestra.run.smithi007.stdout:60129542202 2024-09-12T09:33:45.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542201 got 60129542201 for osd.1 2024-09-12T09:33:45.036 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542202 got 60129542202 for osd.5 2024-09-12T09:33:45.057 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.060 INFO:teuthology.orchestra.run.smithi007.stdout:60129542203 2024-09-12T09:33:45.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542203 for osd.3 2024-09-12T09:33:45.077 DEBUG:teuthology.parallel:result is None 2024-09-12T09:33:45.077 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-12T09:33:45.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:45.424 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-12T09:33:44.045706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20186,"ondisk_log_size":20186,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372624,"kb_used_data":22356,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602096,"statfs":{"total":773094113280,"available":772712546304,"internally_reserved":0,"allocated":22892544,"data_stored":12446446,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.340460"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'650","reported_seq":5275,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152988+0000","last_change":"2024-09-12T09:33:26.738017+0000","last_active":"2024-09-12T09:33:42.152988+0000","last_peered":"2024-09-12T09:33:42.152988+0000","last_clean":"2024-09-12T09:33:42.152988+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:42.152988+0000","last_undegraded":"2024-09-12T09:33:42.152988+0000","last_fullsized":"2024-09-12T09:33:42.152988+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:33.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"220'419","reported_seq":7359,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152863+0000","last_change":"2024-09-12T09:33:25.850791+0000","last_active":"2024-09-12T09:33:42.152863+0000","last_peered":"2024-09-12T09:33:42.152863+0000","last_clean":"2024-09-12T09:33:42.152863+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:42.152863+0000","last_undegraded":"2024-09-12T09:33:42.152863+0000","last_fullsized":"2024-09-12T09:33:42.152863+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:43.891242+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154403+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154403+0000","last_peered":"2024-09-12T09:33:42.154403+0000","last_clean":"2024-09-12T09:33:42.154403+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154403+0000","last_undegraded":"2024-09-12T09:33:42.154403+0000","last_fullsized":"2024-09-12T09:33:42.154403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"209'442","reported_seq":4195,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154438+0000","last_change":"2024-09-12T09:33:32.791542+0000","last_active":"2024-09-12T09:33:42.154438+0000","last_peered":"2024-09-12T09:33:42.154438+0000","last_clean":"2024-09-12T09:33:42.154438+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:42.154438+0000","last_undegraded":"2024-09-12T09:33:42.154438+0000","last_fullsized":"2024-09-12T09:33:42.154438+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:05.583632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8224,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154311+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154311+0000","last_peered":"2024-09-12T09:33:42.154311+0000","last_clean":"2024-09-12T09:33:42.154311+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154311+0000","last_undegraded":"2024-09-12T09:33:42.154311+0000","last_fullsized":"2024-09-12T09:33:42.154311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:15.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154513+0000","last_change":"2024-09-12T09:33:22.714223+0000","last_active":"2024-09-12T09:33:42.154513+0000","last_peered":"2024-09-12T09:33:42.154513+0000","last_clean":"2024-09-12T09:33:42.154513+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:42.154513+0000","last_undegraded":"2024-09-12T09:33:42.154513+0000","last_fullsized":"2024-09-12T09:33:42.154513+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:21.405138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14340,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860308+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860308+0000","last_peered":"2024-09-12T09:33:41.860308+0000","last_clean":"2024-09-12T09:33:41.860308+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860308+0000","last_undegraded":"2024-09-12T09:33:41.860308+0000","last_fullsized":"2024-09-12T09:33:41.860308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6814,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153266+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153266+0000","last_peered":"2024-09-12T09:33:42.153266+0000","last_clean":"2024-09-12T09:33:42.153266+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153266+0000","last_undegraded":"2024-09-12T09:33:42.153266+0000","last_fullsized":"2024-09-12T09:33:42.153266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5266,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154222+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154222+0000","last_peered":"2024-09-12T09:33:42.154222+0000","last_clean":"2024-09-12T09:33:42.154222+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154222+0000","last_undegraded":"2024-09-12T09:33:42.154222+0000","last_fullsized":"2024-09-12T09:33:42.154222+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":452,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155346+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:33:42.155346+0000","last_peered":"2024-09-12T09:33:42.155346+0000","last_clean":"2024-09-12T09:33:42.155346+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:42.155346+0000","last_undegraded":"2024-09-12T09:33:42.155346+0000","last_fullsized":"2024-09-12T09:33:42.155346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20111,"ondisk_log_size":20111,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2842624,"data_stored":1403010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:45.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:45.719 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:45.737 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-12T09:33:44.045706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20186,"ondisk_log_size":20186,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372624,"kb_used_data":22356,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602096,"statfs":{"total":773094113280,"available":772712546304,"internally_reserved":0,"allocated":22892544,"data_stored":12446446,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.340460"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'650","reported_seq":5275,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152988+0000","last_change":"2024-09-12T09:33:26.738017+0000","last_active":"2024-09-12T09:33:42.152988+0000","last_peered":"2024-09-12T09:33:42.152988+0000","last_clean":"2024-09-12T09:33:42.152988+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:42.152988+0000","last_undegraded":"2024-09-12T09:33:42.152988+0000","last_fullsized":"2024-09-12T09:33:42.152988+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:33.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"220'419","reported_seq":7359,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152863+0000","last_change":"2024-09-12T09:33:25.850791+0000","last_active":"2024-09-12T09:33:42.152863+0000","last_peered":"2024-09-12T09:33:42.152863+0000","last_clean":"2024-09-12T09:33:42.152863+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:42.152863+0000","last_undegraded":"2024-09-12T09:33:42.152863+0000","last_fullsized":"2024-09-12T09:33:42.152863+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:43.891242+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154403+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154403+0000","last_peered":"2024-09-12T09:33:42.154403+0000","last_clean":"2024-09-12T09:33:42.154403+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154403+0000","last_undegraded":"2024-09-12T09:33:42.154403+0000","last_fullsized":"2024-09-12T09:33:42.154403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"209'442","reported_seq":4195,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154438+0000","last_change":"2024-09-12T09:33:32.791542+0000","last_active":"2024-09-12T09:33:42.154438+0000","last_peered":"2024-09-12T09:33:42.154438+0000","last_clean":"2024-09-12T09:33:42.154438+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:42.154438+0000","last_undegraded":"2024-09-12T09:33:42.154438+0000","last_fullsized":"2024-09-12T09:33:42.154438+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:05.583632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8224,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154311+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154311+0000","last_peered":"2024-09-12T09:33:42.154311+0000","last_clean":"2024-09-12T09:33:42.154311+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154311+0000","last_undegraded":"2024-09-12T09:33:42.154311+0000","last_fullsized":"2024-09-12T09:33:42.154311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:15.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154513+0000","last_change":"2024-09-12T09:33:22.714223+0000","last_active":"2024-09-12T09:33:42.154513+0000","last_peered":"2024-09-12T09:33:42.154513+0000","last_clean":"2024-09-12T09:33:42.154513+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:42.154513+0000","last_undegraded":"2024-09-12T09:33:42.154513+0000","last_fullsized":"2024-09-12T09:33:42.154513+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:21.405138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14340,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860308+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860308+0000","last_peered":"2024-09-12T09:33:41.860308+0000","last_clean":"2024-09-12T09:33:41.860308+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860308+0000","last_undegraded":"2024-09-12T09:33:41.860308+0000","last_fullsized":"2024-09-12T09:33:41.860308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6814,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153266+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153266+0000","last_peered":"2024-09-12T09:33:42.153266+0000","last_clean":"2024-09-12T09:33:42.153266+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153266+0000","last_undegraded":"2024-09-12T09:33:42.153266+0000","last_fullsized":"2024-09-12T09:33:42.153266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5266,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154222+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154222+0000","last_peered":"2024-09-12T09:33:42.154222+0000","last_clean":"2024-09-12T09:33:42.154222+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154222+0000","last_undegraded":"2024-09-12T09:33:42.154222+0000","last_fullsized":"2024-09-12T09:33:42.154222+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":452,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155346+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:33:42.155346+0000","last_peered":"2024-09-12T09:33:42.155346+0000","last_clean":"2024-09-12T09:33:42.155346+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:42.155346+0000","last_undegraded":"2024-09-12T09:33:42.155346+0000","last_fullsized":"2024-09-12T09:33:42.155346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20111,"ondisk_log_size":20111,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2842624,"data_stored":1403010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:45.743 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-12T09:33:45.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-12T09:33:46.057 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T09:33:46.067 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T09:33:46.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:46.375 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-12T09:33:46.051462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20186,"ondisk_log_size":20186,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372624,"kb_used_data":22356,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602096,"statfs":{"total":773094113280,"available":772712546304,"internally_reserved":0,"allocated":22892544,"data_stored":12446446,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.213909"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'650","reported_seq":5275,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152988+0000","last_change":"2024-09-12T09:33:26.738017+0000","last_active":"2024-09-12T09:33:42.152988+0000","last_peered":"2024-09-12T09:33:42.152988+0000","last_clean":"2024-09-12T09:33:42.152988+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:42.152988+0000","last_undegraded":"2024-09-12T09:33:42.152988+0000","last_fullsized":"2024-09-12T09:33:42.152988+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:33.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"220'419","reported_seq":7359,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152863+0000","last_change":"2024-09-12T09:33:25.850791+0000","last_active":"2024-09-12T09:33:42.152863+0000","last_peered":"2024-09-12T09:33:42.152863+0000","last_clean":"2024-09-12T09:33:42.152863+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:42.152863+0000","last_undegraded":"2024-09-12T09:33:42.152863+0000","last_fullsized":"2024-09-12T09:33:42.152863+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:43.891242+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154403+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154403+0000","last_peered":"2024-09-12T09:33:42.154403+0000","last_clean":"2024-09-12T09:33:42.154403+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154403+0000","last_undegraded":"2024-09-12T09:33:42.154403+0000","last_fullsized":"2024-09-12T09:33:42.154403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"209'442","reported_seq":4195,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154438+0000","last_change":"2024-09-12T09:33:32.791542+0000","last_active":"2024-09-12T09:33:42.154438+0000","last_peered":"2024-09-12T09:33:42.154438+0000","last_clean":"2024-09-12T09:33:42.154438+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:42.154438+0000","last_undegraded":"2024-09-12T09:33:42.154438+0000","last_fullsized":"2024-09-12T09:33:42.154438+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:05.583632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8224,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154311+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154311+0000","last_peered":"2024-09-12T09:33:42.154311+0000","last_clean":"2024-09-12T09:33:42.154311+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154311+0000","last_undegraded":"2024-09-12T09:33:42.154311+0000","last_fullsized":"2024-09-12T09:33:42.154311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:15.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154513+0000","last_change":"2024-09-12T09:33:22.714223+0000","last_active":"2024-09-12T09:33:42.154513+0000","last_peered":"2024-09-12T09:33:42.154513+0000","last_clean":"2024-09-12T09:33:42.154513+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:42.154513+0000","last_undegraded":"2024-09-12T09:33:42.154513+0000","last_fullsized":"2024-09-12T09:33:42.154513+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:21.405138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14340,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860308+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860308+0000","last_peered":"2024-09-12T09:33:41.860308+0000","last_clean":"2024-09-12T09:33:41.860308+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860308+0000","last_undegraded":"2024-09-12T09:33:41.860308+0000","last_fullsized":"2024-09-12T09:33:41.860308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6814,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153266+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153266+0000","last_peered":"2024-09-12T09:33:42.153266+0000","last_clean":"2024-09-12T09:33:42.153266+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153266+0000","last_undegraded":"2024-09-12T09:33:42.153266+0000","last_fullsized":"2024-09-12T09:33:42.153266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5266,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154222+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154222+0000","last_peered":"2024-09-12T09:33:42.154222+0000","last_clean":"2024-09-12T09:33:42.154222+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154222+0000","last_undegraded":"2024-09-12T09:33:42.154222+0000","last_fullsized":"2024-09-12T09:33:42.154222+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":452,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155346+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:33:42.155346+0000","last_peered":"2024-09-12T09:33:42.155346+0000","last_clean":"2024-09-12T09:33:42.155346+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:42.155346+0000","last_undegraded":"2024-09-12T09:33:42.155346+0000","last_fullsized":"2024-09-12T09:33:42.155346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20111,"ondisk_log_size":20111,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2842624,"data_stored":1403010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:46.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:46.685 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-12T09:33:46.051462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20186,"ondisk_log_size":20186,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372624,"kb_used_data":22356,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602096,"statfs":{"total":773094113280,"available":772712546304,"internally_reserved":0,"allocated":22892544,"data_stored":12446446,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.213909"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'650","reported_seq":5275,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152988+0000","last_change":"2024-09-12T09:33:26.738017+0000","last_active":"2024-09-12T09:33:42.152988+0000","last_peered":"2024-09-12T09:33:42.152988+0000","last_clean":"2024-09-12T09:33:42.152988+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:42.152988+0000","last_undegraded":"2024-09-12T09:33:42.152988+0000","last_fullsized":"2024-09-12T09:33:42.152988+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:33.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"220'419","reported_seq":7359,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152863+0000","last_change":"2024-09-12T09:33:25.850791+0000","last_active":"2024-09-12T09:33:42.152863+0000","last_peered":"2024-09-12T09:33:42.152863+0000","last_clean":"2024-09-12T09:33:42.152863+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:42.152863+0000","last_undegraded":"2024-09-12T09:33:42.152863+0000","last_fullsized":"2024-09-12T09:33:42.152863+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:43.891242+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154403+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154403+0000","last_peered":"2024-09-12T09:33:42.154403+0000","last_clean":"2024-09-12T09:33:42.154403+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154403+0000","last_undegraded":"2024-09-12T09:33:42.154403+0000","last_fullsized":"2024-09-12T09:33:42.154403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"209'442","reported_seq":4195,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154438+0000","last_change":"2024-09-12T09:33:32.791542+0000","last_active":"2024-09-12T09:33:42.154438+0000","last_peered":"2024-09-12T09:33:42.154438+0000","last_clean":"2024-09-12T09:33:42.154438+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:42.154438+0000","last_undegraded":"2024-09-12T09:33:42.154438+0000","last_fullsized":"2024-09-12T09:33:42.154438+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:05.583632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8224,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154311+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154311+0000","last_peered":"2024-09-12T09:33:42.154311+0000","last_clean":"2024-09-12T09:33:42.154311+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154311+0000","last_undegraded":"2024-09-12T09:33:42.154311+0000","last_fullsized":"2024-09-12T09:33:42.154311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:15.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154513+0000","last_change":"2024-09-12T09:33:22.714223+0000","last_active":"2024-09-12T09:33:42.154513+0000","last_peered":"2024-09-12T09:33:42.154513+0000","last_clean":"2024-09-12T09:33:42.154513+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:42.154513+0000","last_undegraded":"2024-09-12T09:33:42.154513+0000","last_fullsized":"2024-09-12T09:33:42.154513+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:21.405138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14340,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860308+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860308+0000","last_peered":"2024-09-12T09:33:41.860308+0000","last_clean":"2024-09-12T09:33:41.860308+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860308+0000","last_undegraded":"2024-09-12T09:33:41.860308+0000","last_fullsized":"2024-09-12T09:33:41.860308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6814,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153266+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153266+0000","last_peered":"2024-09-12T09:33:42.153266+0000","last_clean":"2024-09-12T09:33:42.153266+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153266+0000","last_undegraded":"2024-09-12T09:33:42.153266+0000","last_fullsized":"2024-09-12T09:33:42.153266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5266,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154222+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154222+0000","last_peered":"2024-09-12T09:33:42.154222+0000","last_clean":"2024-09-12T09:33:42.154222+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154222+0000","last_undegraded":"2024-09-12T09:33:42.154222+0000","last_fullsized":"2024-09-12T09:33:42.154222+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":452,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155346+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:33:42.155346+0000","last_peered":"2024-09-12T09:33:42.155346+0000","last_clean":"2024-09-12T09:33:42.155346+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:42.155346+0000","last_undegraded":"2024-09-12T09:33:42.155346+0000","last_fullsized":"2024-09-12T09:33:42.155346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20111,"ondisk_log_size":20111,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2842624,"data_stored":1403010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:46.687 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T09:33:46.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:46.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:46.984 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:47.000 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-12T09:33:46.051462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20186,"ondisk_log_size":20186,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372624,"kb_used_data":22356,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602096,"statfs":{"total":773094113280,"available":772712546304,"internally_reserved":0,"allocated":22892544,"data_stored":12446446,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.213909"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"211'650","reported_seq":5275,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152988+0000","last_change":"2024-09-12T09:33:26.738017+0000","last_active":"2024-09-12T09:33:42.152988+0000","last_peered":"2024-09-12T09:33:42.152988+0000","last_clean":"2024-09-12T09:33:42.152988+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:42.152988+0000","last_undegraded":"2024-09-12T09:33:42.152988+0000","last_fullsized":"2024-09-12T09:33:42.152988+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:33.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"220'419","reported_seq":7359,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152863+0000","last_change":"2024-09-12T09:33:25.850791+0000","last_active":"2024-09-12T09:33:42.152863+0000","last_peered":"2024-09-12T09:33:42.152863+0000","last_clean":"2024-09-12T09:33:42.152863+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:42.152863+0000","last_undegraded":"2024-09-12T09:33:42.152863+0000","last_fullsized":"2024-09-12T09:33:42.152863+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:43.891242+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154403+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154403+0000","last_peered":"2024-09-12T09:33:42.154403+0000","last_clean":"2024-09-12T09:33:42.154403+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154403+0000","last_undegraded":"2024-09-12T09:33:42.154403+0000","last_fullsized":"2024-09-12T09:33:42.154403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:35.850578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"209'442","reported_seq":4195,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154438+0000","last_change":"2024-09-12T09:33:32.791542+0000","last_active":"2024-09-12T09:33:42.154438+0000","last_peered":"2024-09-12T09:33:42.154438+0000","last_clean":"2024-09-12T09:33:42.154438+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:42.154438+0000","last_undegraded":"2024-09-12T09:33:42.154438+0000","last_fullsized":"2024-09-12T09:33:42.154438+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:05.583632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8224,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154311+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154311+0000","last_peered":"2024-09-12T09:33:42.154311+0000","last_clean":"2024-09-12T09:33:42.154311+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154311+0000","last_undegraded":"2024-09-12T09:33:42.154311+0000","last_fullsized":"2024-09-12T09:33:42.154311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:15.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":247,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154513+0000","last_change":"2024-09-12T09:33:22.714223+0000","last_active":"2024-09-12T09:33:42.154513+0000","last_peered":"2024-09-12T09:33:42.154513+0000","last_clean":"2024-09-12T09:33:42.154513+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:42.154513+0000","last_undegraded":"2024-09-12T09:33:42.154513+0000","last_fullsized":"2024-09-12T09:33:42.154513+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:19:21.405138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14340,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860308+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860308+0000","last_peered":"2024-09-12T09:33:41.860308+0000","last_clean":"2024-09-12T09:33:41.860308+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860308+0000","last_undegraded":"2024-09-12T09:33:41.860308+0000","last_fullsized":"2024-09-12T09:33:41.860308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6814,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153266+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153266+0000","last_peered":"2024-09-12T09:33:42.153266+0000","last_clean":"2024-09-12T09:33:42.153266+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153266+0000","last_undegraded":"2024-09-12T09:33:42.153266+0000","last_fullsized":"2024-09-12T09:33:42.153266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5266,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154222+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154222+0000","last_peered":"2024-09-12T09:33:42.154222+0000","last_clean":"2024-09-12T09:33:42.154222+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154222+0000","last_undegraded":"2024-09-12T09:33:42.154222+0000","last_fullsized":"2024-09-12T09:33:42.154222+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":452,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155346+0000","last_change":"2024-09-12T09:29:25.950437+0000","last_active":"2024-09-12T09:33:42.155346+0000","last_peered":"2024-09-12T09:33:42.155346+0000","last_clean":"2024-09-12T09:33:42.155346+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:42.155346+0000","last_undegraded":"2024-09-12T09:33:42.155346+0000","last_fullsized":"2024-09-12T09:33:42.155346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:37.385162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20111,"ondisk_log_size":20111,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2842624,"data_stored":1403010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542202,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47864,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323976,"statfs":{"total":96636764160,"available":96587751424,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935942,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:47.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T09:33:47.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:47.310 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":236,"fsid":"177484da-1bae-43aa-8977-8d19d52d569f","created":"2024-09-12T09:29:02.966061+0000","modified":"2024-09-12T09:33:41.851479+0000","last_up_change":"2024-09-12T09:33:22.142642+0000","last_in_change":"2024-09-12T09:31:51.831699+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T09:29:16.007145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T09:29:22.198256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-12T09:29:32.744560+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":55,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37709081-75a3-4326-93b5-5598d715ea86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":140,"up_from":216,"up_thru":220,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6817","nonce":1479059197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6819","nonce":1479059197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6823","nonce":1479059197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":1479059197},{"type":"v1","addr":"172.21.15.7:6821","nonce":1479059197}]},"public_addr":"172.21.15.7:6817/1479059197","cluster_addr":"172.21.15.7:6819/1479059197","heartbeat_back_addr":"172.21.15.7:6823/1479059197","heartbeat_front_addr":"172.21.15.7:6821/1479059197","state":["exists","up"]},{"osd":1,"uuid":"d25b680f-1839-4b35-8ad0-ae6bed56608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6825","nonce":2164663219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6827","nonce":2164663219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6831","nonce":2164663219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":2164663219},{"type":"v1","addr":"172.21.15.7:6829","nonce":2164663219}]},"public_addr":"172.21.15.7:6825/2164663219","cluster_addr":"172.21.15.7:6827/2164663219","heartbeat_back_addr":"172.21.15.7:6831/2164663219","heartbeat_front_addr":"172.21.15.7:6829/2164663219","state":["exists","up"]},{"osd":2,"uuid":"e209e2d0-0699-4641-a01e-462f5d295d7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6801","nonce":139751272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6803","nonce":139751272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6807","nonce":139751272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":139751272},{"type":"v1","addr":"172.21.15.7:6805","nonce":139751272}]},"public_addr":"172.21.15.7:6801/139751272","cluster_addr":"172.21.15.7:6803/139751272","heartbeat_back_addr":"172.21.15.7:6807/139751272","heartbeat_front_addr":"172.21.15.7:6805/139751272","state":["exists","up"]},{"osd":3,"uuid":"f00610a7-05b5-479e-ac16-f9a2b8a4ec1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":226,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6809","nonce":4287683774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6811","nonce":4287683774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6815","nonce":4287683774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4287683774},{"type":"v1","addr":"172.21.15.7:6813","nonce":4287683774}]},"public_addr":"172.21.15.7:6809/4287683774","cluster_addr":"172.21.15.7:6811/4287683774","heartbeat_back_addr":"172.21.15.7:6815/4287683774","heartbeat_front_addr":"172.21.15.7:6813/4287683774","state":["exists","up"]},{"osd":4,"uuid":"85af91a4-b30a-47d4-a8cf-4a40e0c18869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6827","nonce":1982719715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6829","nonce":1982719715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6832","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6833","nonce":1982719715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":1982719715},{"type":"v1","addr":"172.21.15.115:6831","nonce":1982719715}]},"public_addr":"172.21.15.115:6827/1982719715","cluster_addr":"172.21.15.115:6829/1982719715","heartbeat_back_addr":"172.21.15.115:6833/1982719715","heartbeat_front_addr":"172.21.15.115:6831/1982719715","state":["exists","up"]},{"osd":5,"uuid":"a11aa028-c5a6-4b28-ad9e-8a24028273cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6801","nonce":1321805872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6803","nonce":1321805872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6807","nonce":1321805872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":1321805872},{"type":"v1","addr":"172.21.15.115:6805","nonce":1321805872}]},"public_addr":"172.21.15.115:6801/1321805872","cluster_addr":"172.21.15.115:6803/1321805872","heartbeat_back_addr":"172.21.15.115:6807/1321805872","heartbeat_front_addr":"172.21.15.115:6805/1321805872","state":["exists","up"]},{"osd":6,"uuid":"78df53b3-cb81-420f-8657-bf2c51381f54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6809","nonce":3633441839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6811","nonce":3633441839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6815","nonce":3633441839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":3633441839},{"type":"v1","addr":"172.21.15.115:6813","nonce":3633441839}]},"public_addr":"172.21.15.115:6809/3633441839","cluster_addr":"172.21.15.115:6811/3633441839","heartbeat_back_addr":"172.21.15.115:6815/3633441839","heartbeat_front_addr":"172.21.15.115:6813/3633441839","state":["exists","up"]},{"osd":7,"uuid":"393284af-2f90-46be-99a0-15f9015c3e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6817","nonce":767353918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6819","nonce":767353918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6823","nonce":767353918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":767353918},{"type":"v1","addr":"172.21.15.115:6821","nonce":767353918}]},"public_addr":"172.21.15.115:6817/767353918","cluster_addr":"172.21.15.115:6819/767353918","heartbeat_back_addr":"172.21.15.115:6823/767353918","heartbeat_front_addr":"172.21.15.115:6821/767353918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.165826+0000","dead_epoch":141},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.541881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.550164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.611505+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.331875+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.389036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:12.613814+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T09:29:13.178370+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[2,6]},{"pgid":"3.2","osds":[0,6]},{"pgid":"3.e","osds":[5,0]},{"pgid":"3.12","osds":[7,2]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T09:33:48.331 INFO:tasks.ceph:Scrubbing osd.0 2024-09-12T09:33:48.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:48.496 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:48.496 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-12T09:33:48.496 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:48.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-12T09:33:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T09:33:48.829 INFO:tasks.ceph:Scrubbing osd.1 2024-09-12T09:33:48.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:48.969 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:48.970 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:48.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-12T09:33:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T09:33:49.300 INFO:tasks.ceph:Scrubbing osd.2 2024-09-12T09:33:49.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:49.454 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:49.455 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:49.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-12T09:33:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T09:33:49.789 INFO:tasks.ceph:Scrubbing osd.3 2024-09-12T09:33:49.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:49.928 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:49.928 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:49.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-12T09:33:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-12T09:33:50.248 INFO:tasks.ceph:Scrubbing osd.4 2024-09-12T09:33:50.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:50.405 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:50.406 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:50.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-12T09:33:50.715 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-12T09:33:50.733 INFO:tasks.ceph:Scrubbing osd.5 2024-09-12T09:33:50.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:50.890 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:50.891 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:50.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-12T09:33:51.202 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-12T09:33:51.220 INFO:tasks.ceph:Scrubbing osd.6 2024-09-12T09:33:51.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:51.363 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:51.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-12T09:33:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-12T09:33:51.698 INFO:tasks.ceph:Scrubbing osd.7 2024-09-12T09:33:51.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-12T09:33:51.857 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-12T09:33:51.858 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T09:33:51.858 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-12T09:33:51.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-12T09:33:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-12T09:33:52.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:33:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:33:52.508 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:33:52.526 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-12T09:33:52.053394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20202,"ondisk_log_size":20202,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372620,"kb_used_data":22368,"kb_used_omap":51,"kb_used_meta":350028,"kb_avail":754602100,"statfs":{"total":773094113280,"available":772712550400,"internally_reserved":0,"allocated":22904832,"data_stored":12452404,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":358429004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.197455"},"pg_stats":[{"pgid":"3.1f","version":"220'804","reported_seq":6785,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153251+0000","last_change":"2024-09-12T09:33:25.906275+0000","last_active":"2024-09-12T09:33:42.153251+0000","last_peered":"2024-09-12T09:33:42.153251+0000","last_clean":"2024-09-12T09:33:42.153251+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:42.153251+0000","last_undegraded":"2024-09-12T09:33:42.153251+0000","last_fullsized":"2024-09-12T09:33:42.153251+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:25.291617+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"211'758","reported_seq":6469,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154115+0000","last_change":"2024-09-12T09:33:09.941964+0000","last_active":"2024-09-12T09:33:42.154115+0000","last_peered":"2024-09-12T09:33:42.154115+0000","last_clean":"2024-09-12T09:33:42.154115+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:42.154115+0000","last_undegraded":"2024-09-12T09:33:42.154115+0000","last_fullsized":"2024-09-12T09:33:42.154115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:09.780231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'529","reported_seq":8302,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860079+0000","last_change":"2024-09-12T09:33:09.959049+0000","last_active":"2024-09-12T09:33:41.860079+0000","last_peered":"2024-09-12T09:33:41.860079+0000","last_clean":"2024-09-12T09:33:41.860079+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:41.860079+0000","last_undegraded":"2024-09-12T09:33:41.860079+0000","last_fullsized":"2024-09-12T09:33:41.860079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:42:17.177615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'748","reported_seq":7231,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859492+0000","last_change":"2024-09-12T09:33:09.955672+0000","last_active":"2024-09-12T09:33:41.859492+0000","last_peered":"2024-09-12T09:33:41.859492+0000","last_clean":"2024-09-12T09:33:41.859492+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:41.859492+0000","last_undegraded":"2024-09-12T09:33:41.859492+0000","last_fullsized":"2024-09-12T09:33:41.859492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:05.370784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"212'393","reported_seq":5903,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152721+0000","last_change":"2024-09-12T09:33:23.821487+0000","last_active":"2024-09-12T09:33:42.152721+0000","last_peered":"2024-09-12T09:33:42.152721+0000","last_clean":"2024-09-12T09:33:42.152721+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:42.152721+0000","last_undegraded":"2024-09-12T09:33:42.152721+0000","last_fullsized":"2024-09-12T09:33:42.152721+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:33:44.361931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'591","reported_seq":9705,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152875+0000","last_change":"2024-09-12T09:33:09.940230+0000","last_active":"2024-09-12T09:33:42.152875+0000","last_peered":"2024-09-12T09:33:42.152875+0000","last_clean":"2024-09-12T09:33:42.152875+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:42.152875+0000","last_undegraded":"2024-09-12T09:33:42.152875+0000","last_fullsized":"2024-09-12T09:33:42.152875+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:53.819811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"212'446","reported_seq":4090,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154217+0000","last_change":"2024-09-12T09:33:09.949808+0000","last_active":"2024-09-12T09:33:42.154217+0000","last_peered":"2024-09-12T09:33:42.154217+0000","last_clean":"2024-09-12T09:33:42.154217+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:42.154217+0000","last_undegraded":"2024-09-12T09:33:42.154217+0000","last_fullsized":"2024-09-12T09:33:42.154217+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:19:15.449331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"211'478","reported_seq":4416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152441+0000","last_change":"2024-09-12T09:33:24.830448+0000","last_active":"2024-09-12T09:33:42.152441+0000","last_peered":"2024-09-12T09:33:42.152441+0000","last_clean":"2024-09-12T09:33:42.152441+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:42.152441+0000","last_undegraded":"2024-09-12T09:33:42.152441+0000","last_fullsized":"2024-09-12T09:33:42.152441+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:34.306407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"236'654","reported_seq":5287,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:49.507530+0000","last_change":"2024-09-12T09:33:49.507530+0000","last_active":"2024-09-12T09:33:49.507530+0000","last_peered":"2024-09-12T09:33:49.507530+0000","last_clean":"2024-09-12T09:33:49.507530+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:49.507530+0000","last_undegraded":"2024-09-12T09:33:49.507530+0000","last_fullsized":"2024-09-12T09:33:49.507530+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'654","last_scrub_stamp":"2024-09-12T09:33:49.507431+0000","last_deep_scrub":"236'654","last_deep_scrub_stamp":"2024-09-12T09:33:49.507431+0000","last_clean_scrub_stamp":"2024-09-12T09:33:49.507431+0000","objects_scrubbed":4,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:51.086594+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"212'557","reported_seq":16012,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860852+0000","last_change":"2024-09-12T09:33:09.962477+0000","last_active":"2024-09-12T09:33:41.860852+0000","last_peered":"2024-09-12T09:33:41.860852+0000","last_clean":"2024-09-12T09:33:41.860852+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:41.860852+0000","last_undegraded":"2024-09-12T09:33:41.860852+0000","last_fullsized":"2024-09-12T09:33:41.860852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:20.349023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'831","reported_seq":8895,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860985+0000","last_change":"2024-09-12T09:33:10.031379+0000","last_active":"2024-09-12T09:33:41.860985+0000","last_peered":"2024-09-12T09:33:41.860985+0000","last_clean":"2024-09-12T09:33:41.860985+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:41.860985+0000","last_undegraded":"2024-09-12T09:33:41.860985+0000","last_fullsized":"2024-09-12T09:33:41.860985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:50.473203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"212'868","reported_seq":12502,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153198+0000","last_change":"2024-09-12T09:33:09.942180+0000","last_active":"2024-09-12T09:33:42.153198+0000","last_peered":"2024-09-12T09:33:42.153198+0000","last_clean":"2024-09-12T09:33:42.153198+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:42.153198+0000","last_undegraded":"2024-09-12T09:33:42.153198+0000","last_fullsized":"2024-09-12T09:33:42.153198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:24:49.437145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":130,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859904+0000","last_change":"2024-09-12T09:33:38.793827+0000","last_active":"2024-09-12T09:33:41.859904+0000","last_peered":"2024-09-12T09:33:41.859904+0000","last_clean":"2024-09-12T09:33:41.859904+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:41.859904+0000","last_undegraded":"2024-09-12T09:33:41.859904+0000","last_fullsized":"2024-09-12T09:33:41.859904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:19.003388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"236'424","reported_seq":7372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.488430+0000","last_change":"2024-09-12T09:33:50.488430+0000","last_active":"2024-09-12T09:33:50.488430+0000","last_peered":"2024-09-12T09:33:50.488430+0000","last_clean":"2024-09-12T09:33:50.488430+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:50.488430+0000","last_undegraded":"2024-09-12T09:33:50.488430+0000","last_fullsized":"2024-09-12T09:33:50.488430+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'424","last_scrub_stamp":"2024-09-12T09:33:50.488356+0000","last_deep_scrub":"236'424","last_deep_scrub_stamp":"2024-09-12T09:33:50.488356+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.488356+0000","objects_scrubbed":5,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:59.989965+0000","scrub_duration":3,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":233,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154519+0000","last_change":"2024-09-12T09:31:39.763696+0000","last_active":"2024-09-12T09:33:42.154519+0000","last_peered":"2024-09-12T09:33:42.154519+0000","last_clean":"2024-09-12T09:33:42.154519+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:42.154519+0000","last_undegraded":"2024-09-12T09:33:42.154519+0000","last_fullsized":"2024-09-12T09:33:42.154519+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"236'449","reported_seq":4210,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.749691+0000","last_change":"2024-09-12T09:33:50.749691+0000","last_active":"2024-09-12T09:33:50.749691+0000","last_peered":"2024-09-12T09:33:50.749691+0000","last_clean":"2024-09-12T09:33:50.749691+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:50.749691+0000","last_undegraded":"2024-09-12T09:33:50.749691+0000","last_fullsized":"2024-09-12T09:33:50.749691+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'449","last_scrub_stamp":"2024-09-12T09:33:50.749572+0000","last_deep_scrub":"236'449","last_deep_scrub_stamp":"2024-09-12T09:33:50.749572+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.749572+0000","objects_scrubbed":7,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:56.817190+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860728+0000","last_change":"2024-09-12T09:33:22.714428+0000","last_active":"2024-09-12T09:33:41.860728+0000","last_peered":"2024-09-12T09:33:41.860728+0000","last_clean":"2024-09-12T09:33:41.860728+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:41.860728+0000","last_undegraded":"2024-09-12T09:33:41.860728+0000","last_fullsized":"2024-09-12T09:33:41.860728+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:07:38.840914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"209'1145","reported_seq":8225,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154430+0000","last_change":"2024-09-12T09:33:09.950909+0000","last_active":"2024-09-12T09:33:42.154430+0000","last_peered":"2024-09-12T09:33:42.154430+0000","last_clean":"2024-09-12T09:33:42.154430+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:42.154430+0000","last_undegraded":"2024-09-12T09:33:42.154430+0000","last_fullsized":"2024-09-12T09:33:42.154430+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1145,"log_dups_size":0,"ondisk_log_size":1145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":256,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.248053+0000","last_change":"2024-09-12T09:33:50.248053+0000","last_active":"2024-09-12T09:33:50.248053+0000","last_peered":"2024-09-12T09:33:50.248053+0000","last_clean":"2024-09-12T09:33:50.248053+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:50.248053+0000","last_undegraded":"2024-09-12T09:33:50.248053+0000","last_fullsized":"2024-09-12T09:33:50.248053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:50.247987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:50.247987+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.247987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:10:25.406300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"212'296","reported_seq":14341,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860416+0000","last_change":"2024-09-12T09:33:09.961064+0000","last_active":"2024-09-12T09:33:41.860416+0000","last_peered":"2024-09-12T09:33:41.860416+0000","last_clean":"2024-09-12T09:33:41.860416+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:41.860416+0000","last_undegraded":"2024-09-12T09:33:41.860416+0000","last_fullsized":"2024-09-12T09:33:41.860416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.550349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"212'1214","reported_seq":6815,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153385+0000","last_change":"2024-09-12T09:33:25.717843+0000","last_active":"2024-09-12T09:33:42.153385+0000","last_peered":"2024-09-12T09:33:42.153385+0000","last_clean":"2024-09-12T09:33:42.153385+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:42.153385+0000","last_undegraded":"2024-09-12T09:33:42.153385+0000","last_fullsized":"2024-09-12T09:33:42.153385+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:36.467950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":491,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859167+0000","last_change":"2024-09-12T09:31:39.769298+0000","last_active":"2024-09-12T09:33:41.859167+0000","last_peered":"2024-09-12T09:33:41.859167+0000","last_clean":"2024-09-12T09:33:41.859167+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:41.859167+0000","last_undegraded":"2024-09-12T09:33:41.859167+0000","last_fullsized":"2024-09-12T09:33:41.859167+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.090761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"211'414","reported_seq":5267,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.154275+0000","last_change":"2024-09-12T09:33:09.949811+0000","last_active":"2024-09-12T09:33:42.154275+0000","last_peered":"2024-09-12T09:33:42.154275+0000","last_clean":"2024-09-12T09:33:42.154275+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:42.154275+0000","last_undegraded":"2024-09-12T09:33:42.154275+0000","last_fullsized":"2024-09-12T09:33:42.154275+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:17:14.281711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":460,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.755055+0000","last_change":"2024-09-12T09:33:50.755055+0000","last_active":"2024-09-12T09:33:50.755055+0000","last_peered":"2024-09-12T09:33:50.755055+0000","last_clean":"2024-09-12T09:33:50.755055+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:50.755055+0000","last_undegraded":"2024-09-12T09:33:50.755055+0000","last_fullsized":"2024-09-12T09:33:50.755055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:50.754913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:50.754913+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.754913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:58.495614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"209'532","reported_seq":13564,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.155095+0000","last_change":"2024-09-12T09:33:09.937555+0000","last_active":"2024-09-12T09:33:42.155095+0000","last_peered":"2024-09-12T09:33:42.155095+0000","last_clean":"2024-09-12T09:33:42.155095+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:42.155095+0000","last_undegraded":"2024-09-12T09:33:42.155095+0000","last_fullsized":"2024-09-12T09:33:42.155095+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:54:43.824595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":361,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859172+0000","last_change":"2024-09-12T09:31:39.764171+0000","last_active":"2024-09-12T09:33:41.859172+0000","last_peered":"2024-09-12T09:33:41.859172+0000","last_clean":"2024-09-12T09:33:41.859172+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:41.859172+0000","last_undegraded":"2024-09-12T09:33:41.859172+0000","last_fullsized":"2024-09-12T09:33:41.859172+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:03.268542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"211'417","reported_seq":6028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152640+0000","last_change":"2024-09-12T09:33:26.738441+0000","last_active":"2024-09-12T09:33:42.152640+0000","last_peered":"2024-09-12T09:33:42.152640+0000","last_clean":"2024-09-12T09:33:42.152640+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:42.152640+0000","last_undegraded":"2024-09-12T09:33:42.152640+0000","last_fullsized":"2024-09-12T09:33:42.152640+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:39.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":450,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859418+0000","last_change":"2024-09-12T09:29:25.963671+0000","last_active":"2024-09-12T09:33:41.859418+0000","last_peered":"2024-09-12T09:33:41.859418+0000","last_clean":"2024-09-12T09:33:41.859418+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:41.859418+0000","last_undegraded":"2024-09-12T09:33:41.859418+0000","last_fullsized":"2024-09-12T09:33:41.859418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:22.916112+0000","last_clean_scrub_stamp":"2024-09-12T09:29:22.916112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:25:09.437389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.858888+0000","last_change":"2024-09-12T09:33:22.725439+0000","last_active":"2024-09-12T09:33:41.858888+0000","last_peered":"2024-09-12T09:33:41.858888+0000","last_clean":"2024-09-12T09:33:41.858888+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:41.858888+0000","last_undegraded":"2024-09-12T09:33:41.858888+0000","last_fullsized":"2024-09-12T09:33:41.858888+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:16.082807+0000","last_clean_scrub_stamp":"2024-09-12T09:29:16.082807+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:10.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"212'1319","reported_seq":6601,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152581+0000","last_change":"2024-09-12T09:33:09.942046+0000","last_active":"2024-09-12T09:33:42.152581+0000","last_peered":"2024-09-12T09:33:42.152581+0000","last_clean":"2024-09-12T09:33:42.152581+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:42.152581+0000","last_undegraded":"2024-09-12T09:33:42.152581+0000","last_fullsized":"2024-09-12T09:33:42.152581+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:00.746900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"212'727","reported_seq":5170,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.152894+0000","last_change":"2024-09-12T09:33:09.940373+0000","last_active":"2024-09-12T09:33:42.152894+0000","last_peered":"2024-09-12T09:33:42.152894+0000","last_clean":"2024-09-12T09:33:42.152894+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:42.152894+0000","last_undegraded":"2024-09-12T09:33:42.152894+0000","last_fullsized":"2024-09-12T09:33:42.152894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:35.427175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'506","reported_seq":6008,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859598+0000","last_change":"2024-09-12T09:33:38.716073+0000","last_active":"2024-09-12T09:33:41.859598+0000","last_peered":"2024-09-12T09:33:41.859598+0000","last_clean":"2024-09-12T09:33:41.859598+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:41.859598+0000","last_undegraded":"2024-09-12T09:33:41.859598+0000","last_fullsized":"2024-09-12T09:33:41.859598+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:31.945139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"211'513","reported_seq":7078,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:42.153193+0000","last_change":"2024-09-12T09:33:25.717348+0000","last_active":"2024-09-12T09:33:42.153193+0000","last_peered":"2024-09-12T09:33:42.153193+0000","last_clean":"2024-09-12T09:33:42.153193+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:42.153193+0000","last_undegraded":"2024-09-12T09:33:42.153193+0000","last_fullsized":"2024-09-12T09:33:42.153193+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:35.676990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"209'511","reported_seq":4644,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859690+0000","last_change":"2024-09-12T09:33:09.956808+0000","last_active":"2024-09-12T09:33:41.859690+0000","last_peered":"2024-09-12T09:33:41.859690+0000","last_clean":"2024-09-12T09:33:41.859690+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:41.859690+0000","last_undegraded":"2024-09-12T09:33:41.859690+0000","last_fullsized":"2024-09-12T09:33:41.859690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:34.676320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"210'597","reported_seq":9232,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860460+0000","last_change":"2024-09-12T09:33:10.033097+0000","last_active":"2024-09-12T09:33:41.860460+0000","last_peered":"2024-09-12T09:33:41.860460+0000","last_clean":"2024-09-12T09:33:41.860460+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:41.860460+0000","last_undegraded":"2024-09-12T09:33:41.860460+0000","last_fullsized":"2024-09-12T09:33:41.860460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:33.047654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"221'626","reported_seq":6510,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.860583+0000","last_change":"2024-09-12T09:33:26.948339+0000","last_active":"2024-09-12T09:33:41.860583+0000","last_peered":"2024-09-12T09:33:41.860583+0000","last_clean":"2024-09-12T09:33:41.860583+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:41.860583+0000","last_undegraded":"2024-09-12T09:33:41.860583+0000","last_fullsized":"2024-09-12T09:33:41.860583+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:32:35.143301+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"209'699","reported_seq":7639,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.861577+0000","last_change":"2024-09-12T09:33:10.032441+0000","last_active":"2024-09-12T09:33:41.861577+0000","last_peered":"2024-09-12T09:33:41.861577+0000","last_clean":"2024-09-12T09:33:41.861577+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:41.861577+0000","last_undegraded":"2024-09-12T09:33:41.861577+0000","last_fullsized":"2024-09-12T09:33:41.861577+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:02:29.237096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'681","reported_seq":4297,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859096+0000","last_change":"2024-09-12T09:33:38.718423+0000","last_active":"2024-09-12T09:33:41.859096+0000","last_peered":"2024-09-12T09:33:41.859096+0000","last_clean":"2024-09-12T09:33:41.859096+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:41.859096+0000","last_undegraded":"2024-09-12T09:33:41.859096+0000","last_fullsized":"2024-09-12T09:33:41.859096+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:06:13.632555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"211'671","reported_seq":3714,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859249+0000","last_change":"2024-09-12T09:33:10.030671+0000","last_active":"2024-09-12T09:33:41.859249+0000","last_peered":"2024-09-12T09:33:41.859249+0000","last_clean":"2024-09-12T09:33:41.859249+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:41.859249+0000","last_undegraded":"2024-09-12T09:33:41.859249+0000","last_fullsized":"2024-09-12T09:33:41.859249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:11:28.071311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"211'470","reported_seq":13937,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859580+0000","last_change":"2024-09-12T09:33:10.030115+0000","last_active":"2024-09-12T09:33:41.859580+0000","last_peered":"2024-09-12T09:33:41.859580+0000","last_clean":"2024-09-12T09:33:41.859580+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:41.859580+0000","last_undegraded":"2024-09-12T09:33:41.859580+0000","last_fullsized":"2024-09-12T09:33:41.859580+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:29.900231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"208'259","reported_seq":2056,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:41.859917+0000","last_change":"2024-09-12T09:33:10.031565+0000","last_active":"2024-09-12T09:33:41.859917+0000","last_peered":"2024-09-12T09:33:41.859917+0000","last_clean":"2024-09-12T09:33:41.859917+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:33:41.859917+0000","last_undegraded":"2024-09-12T09:33:41.859917+0000","last_fullsized":"2024-09-12T09:33:41.859917+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:29:33.638470+0000","last_clean_scrub_stamp":"2024-09-12T09:29:33.638470+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:49.016035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20127,"ondisk_log_size":20127,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509499,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29816,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342024,"statfs":{"total":96636764160,"available":96606232576,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52024,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319816,"statfs":{"total":96636764160,"available":96583491584,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33984,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337856,"statfs":{"total":96636764160,"available":96601964544,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935944,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:33:52.533 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.534 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.535 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-12T09:29:22.916112+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=22, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T09:29:16.082807+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=16, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.536 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.537 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.538 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.538 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.538 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-12T09:29:33.638470+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=29, tm_sec=33, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=9, tm_min=33, tm_sec=47, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T09:33:52.538 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T09:34:12.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T09:34:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-12T09:34:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-12T09:34:12.884 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-12T09:34:12.059855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":314,"num_read":256499,"num_read_kb":1583760,"num_write":23413,"num_write_kb":854235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5199,"num_bytes_recovered":844886209,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20508,"ondisk_log_size":20508,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":380896,"kb_used_data":22368,"kb_used_omap":51,"kb_used_meta":358348,"kb_avail":754593824,"statfs":{"total":773094113280,"available":772704075776,"internally_reserved":0,"allocated":22904832,"data_stored":12452404,"data_compressed":13308,"data_compressed_allocated":73728,"data_compressed_original":1179648,"omap_allocated":52916,"internal_metadata":366948684},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003925"},"pg_stats":[{"pgid":"3.1f","version":"236'811","reported_seq":6800,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.521877+0000","last_change":"2024-09-12T09:33:53.521877+0000","last_active":"2024-09-12T09:33:53.521877+0000","last_peered":"2024-09-12T09:33:53.521877+0000","last_clean":"2024-09-12T09:33:53.521877+0000","last_became_active":"2024-09-12T09:33:25.719356+0000","last_became_peered":"2024-09-12T09:33:25.719356+0000","last_unstale":"2024-09-12T09:33:53.521877+0000","last_undegraded":"2024-09-12T09:33:53.521877+0000","last_fullsized":"2024-09-12T09:33:53.521877+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'811","last_scrub_stamp":"2024-09-12T09:33:53.521661+0000","last_deep_scrub":"236'811","last_deep_scrub_stamp":"2024-09-12T09:33:53.521661+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.521661+0000","objects_scrubbed":7,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:18:34.503577+0000","scrub_duration":15,"objects_trimmed":28,"snaptrim_duration":0.18427109999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8284,"num_read_kb":95319,"num_write":984,"num_write_kb":47663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":37831476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"236'767","reported_seq":6486,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:54.814856+0000","last_change":"2024-09-12T09:33:54.814856+0000","last_active":"2024-09-12T09:33:54.814856+0000","last_peered":"2024-09-12T09:33:54.814856+0000","last_clean":"2024-09-12T09:33:54.814856+0000","last_became_active":"2024-09-12T09:31:25.689306+0000","last_became_peered":"2024-09-12T09:31:25.689306+0000","last_unstale":"2024-09-12T09:33:54.814856+0000","last_undegraded":"2024-09-12T09:33:54.814856+0000","last_fullsized":"2024-09-12T09:33:54.814856+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'767","last_scrub_stamp":"2024-09-12T09:33:54.814652+0000","last_deep_scrub":"236'767","last_deep_scrub_stamp":"2024-09-12T09:33:54.814652+0000","last_clean_scrub_stamp":"2024-09-12T09:33:54.814652+0000","objects_scrubbed":9,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:20.663164+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0023487920000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8882,"num_read_kb":41441,"num_write":851,"num_write_kb":38769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6624888,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"236'544","reported_seq":8325,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:58.869069+0000","last_change":"2024-09-12T09:33:58.869069+0000","last_active":"2024-09-12T09:33:58.869069+0000","last_peered":"2024-09-12T09:33:58.869069+0000","last_clean":"2024-09-12T09:33:58.869069+0000","last_became_active":"2024-09-12T09:29:34.678153+0000","last_became_peered":"2024-09-12T09:29:34.678153+0000","last_unstale":"2024-09-12T09:33:58.869069+0000","last_undegraded":"2024-09-12T09:33:58.869069+0000","last_fullsized":"2024-09-12T09:33:58.869069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'544","last_scrub_stamp":"2024-09-12T09:33:58.868811+0000","last_deep_scrub":"236'544","last_deep_scrub_stamp":"2024-09-12T09:33:58.868811+0000","last_clean_scrub_stamp":"2024-09-12T09:33:58.868811+0000","objects_scrubbed":15,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:26.799766+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.007643922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":9125,"num_read_kb":78653,"num_write":493,"num_write_kb":43507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"236'758","reported_seq":7249,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:57.908510+0000","last_change":"2024-09-12T09:33:57.908510+0000","last_active":"2024-09-12T09:33:57.908510+0000","last_peered":"2024-09-12T09:33:57.908510+0000","last_clean":"2024-09-12T09:33:57.908510+0000","last_became_active":"2024-09-12T09:29:34.661614+0000","last_became_peered":"2024-09-12T09:29:34.661614+0000","last_unstale":"2024-09-12T09:33:57.908510+0000","last_undegraded":"2024-09-12T09:33:57.908510+0000","last_fullsized":"2024-09-12T09:33:57.908510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'758","last_scrub_stamp":"2024-09-12T09:33:57.908310+0000","last_deep_scrub":"236'758","last_deep_scrub_stamp":"2024-09-12T09:33:57.908310+0000","last_clean_scrub_stamp":"2024-09-12T09:33:57.908310+0000","objects_scrubbed":10,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:31.538212+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.005186403,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":8388,"num_read_kb":52577,"num_write":818,"num_write_kb":16453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"236'400","reported_seq":5918,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.459203+0000","last_change":"2024-09-12T09:33:51.459203+0000","last_active":"2024-09-12T09:33:51.459203+0000","last_peered":"2024-09-12T09:33:51.459203+0000","last_clean":"2024-09-12T09:33:51.459203+0000","last_became_active":"2024-09-12T09:33:23.730950+0000","last_became_peered":"2024-09-12T09:33:23.730950+0000","last_unstale":"2024-09-12T09:33:51.459203+0000","last_undegraded":"2024-09-12T09:33:51.459203+0000","last_fullsized":"2024-09-12T09:33:51.459203+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'400","last_scrub_stamp":"2024-09-12T09:33:51.459099+0000","last_deep_scrub":"236'400","last_deep_scrub_stamp":"2024-09-12T09:33:51.459099+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.459099+0000","objects_scrubbed":7,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:19.197299+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001077586,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6296,"num_read_kb":20885,"num_write":373,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":7604377,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"236'605","reported_seq":9727,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.879603+0000","last_change":"2024-09-12T09:33:53.879603+0000","last_active":"2024-09-12T09:33:53.879603+0000","last_peered":"2024-09-12T09:33:53.879603+0000","last_clean":"2024-09-12T09:33:53.879603+0000","last_became_active":"2024-09-12T09:32:39.058283+0000","last_became_peered":"2024-09-12T09:32:39.058283+0000","last_unstale":"2024-09-12T09:33:53.879603+0000","last_undegraded":"2024-09-12T09:33:53.879603+0000","last_fullsized":"2024-09-12T09:33:53.879603+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'605","last_scrub_stamp":"2024-09-12T09:33:53.879371+0000","last_deep_scrub":"236'605","last_deep_scrub_stamp":"2024-09-12T09:33:53.879371+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.879371+0000","objects_scrubbed":14,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:00:53.172121+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0018306609999999999,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":9122,"num_read_kb":32161,"num_write":621,"num_write_kb":38645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13735170,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"236'455","reported_seq":4108,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:54.267896+0000","last_change":"2024-09-12T09:33:54.267746+0000","last_active":"2024-09-12T09:33:54.267896+0000","last_peered":"2024-09-12T09:33:54.267896+0000","last_clean":"2024-09-12T09:33:54.267896+0000","last_became_active":"2024-09-12T09:31:42.679192+0000","last_became_peered":"2024-09-12T09:31:42.679192+0000","last_unstale":"2024-09-12T09:33:54.267896+0000","last_undegraded":"2024-09-12T09:33:54.267896+0000","last_fullsized":"2024-09-12T09:33:54.267896+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'455","last_scrub_stamp":"2024-09-12T09:33:54.267549+0000","last_deep_scrub":"236'455","last_deep_scrub_stamp":"2024-09-12T09:33:54.267549+0000","last_clean_scrub_stamp":"2024-09-12T09:33:54.267549+0000","objects_scrubbed":9,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:24:13.588154+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.007125195,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4218,"num_read_kb":15190,"num_write":485,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":5400347,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"236'488","reported_seq":4434,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.772320+0000","last_change":"2024-09-12T09:33:53.772320+0000","last_active":"2024-09-12T09:33:53.772320+0000","last_peered":"2024-09-12T09:33:53.772320+0000","last_clean":"2024-09-12T09:33:53.772320+0000","last_became_active":"2024-09-12T09:33:23.750753+0000","last_became_peered":"2024-09-12T09:33:23.750753+0000","last_unstale":"2024-09-12T09:33:53.772320+0000","last_undegraded":"2024-09-12T09:33:53.772320+0000","last_fullsized":"2024-09-12T09:33:53.772320+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'488","last_scrub_stamp":"2024-09-12T09:33:53.772100+0000","last_deep_scrub":"236'488","last_deep_scrub_stamp":"2024-09-12T09:33:53.772100+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.772100+0000","objects_scrubbed":10,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:29:32.164048+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0035317920000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2637,"num_read_kb":10193,"num_write":493,"num_write_kb":8067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":13861659,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"236'654","reported_seq":5287,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:49.507530+0000","last_change":"2024-09-12T09:33:49.507530+0000","last_active":"2024-09-12T09:33:49.507530+0000","last_peered":"2024-09-12T09:33:49.507530+0000","last_clean":"2024-09-12T09:33:49.507530+0000","last_became_active":"2024-09-12T09:33:26.734814+0000","last_became_peered":"2024-09-12T09:33:26.734814+0000","last_unstale":"2024-09-12T09:33:49.507530+0000","last_undegraded":"2024-09-12T09:33:49.507530+0000","last_fullsized":"2024-09-12T09:33:49.507530+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'654","last_scrub_stamp":"2024-09-12T09:33:49.507431+0000","last_deep_scrub":"236'654","last_deep_scrub_stamp":"2024-09-12T09:33:49.507431+0000","last_clean_scrub_stamp":"2024-09-12T09:33:49.507431+0000","objects_scrubbed":4,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:51.086594+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0095070989999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7132,"num_read_kb":53861,"num_write":738,"num_write_kb":39730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":18607743,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"236'565","reported_seq":16028,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:56.889898+0000","last_change":"2024-09-12T09:33:56.889898+0000","last_active":"2024-09-12T09:33:56.889898+0000","last_peered":"2024-09-12T09:33:56.889898+0000","last_clean":"2024-09-12T09:33:56.889898+0000","last_became_active":"2024-09-12T09:30:33.668325+0000","last_became_peered":"2024-09-12T09:30:33.668325+0000","last_unstale":"2024-09-12T09:33:56.889898+0000","last_undegraded":"2024-09-12T09:33:56.889898+0000","last_fullsized":"2024-09-12T09:33:56.889898+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'565","last_scrub_stamp":"2024-09-12T09:33:56.889590+0000","last_deep_scrub":"236'565","last_deep_scrub_stamp":"2024-09-12T09:33:56.889590+0000","last_clean_scrub_stamp":"2024-09-12T09:33:56.889590+0000","objects_scrubbed":8,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:25.948527+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.010259526999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15947,"num_read_kb":96859,"num_write":590,"num_write_kb":40309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":5628655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"236'844","reported_seq":8916,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:59.445842+0000","last_change":"2024-09-12T09:33:59.445842+0000","last_active":"2024-09-12T09:33:59.445842+0000","last_peered":"2024-09-12T09:33:59.445842+0000","last_clean":"2024-09-12T09:33:59.445842+0000","last_became_active":"2024-09-12T09:30:34.669978+0000","last_became_peered":"2024-09-12T09:30:34.669978+0000","last_unstale":"2024-09-12T09:33:59.445842+0000","last_undegraded":"2024-09-12T09:33:59.445842+0000","last_fullsized":"2024-09-12T09:33:59.445842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'844","last_scrub_stamp":"2024-09-12T09:33:59.445559+0000","last_deep_scrub":"236'844","last_deep_scrub_stamp":"2024-09-12T09:33:59.445559+0000","last_clean_scrub_stamp":"2024-09-12T09:33:59.445559+0000","objects_scrubbed":13,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:00.400324+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.001294863,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":9199,"num_read_kb":102719,"num_write":1034,"num_write_kb":47727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":59686726,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"236'877","reported_seq":12519,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.892934+0000","last_change":"2024-09-12T09:33:52.892934+0000","last_active":"2024-09-12T09:33:52.892934+0000","last_peered":"2024-09-12T09:33:52.892934+0000","last_clean":"2024-09-12T09:33:52.892934+0000","last_became_active":"2024-09-12T09:30:33.667522+0000","last_became_peered":"2024-09-12T09:30:33.667522+0000","last_unstale":"2024-09-12T09:33:52.892934+0000","last_undegraded":"2024-09-12T09:33:52.892934+0000","last_fullsized":"2024-09-12T09:33:52.892934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'877","last_scrub_stamp":"2024-09-12T09:33:52.892692+0000","last_deep_scrub":"236'877","last_deep_scrub_stamp":"2024-09-12T09:33:52.892692+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.892692+0000","objects_scrubbed":9,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:53.903239+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0028122300000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13364,"num_read_kb":53991,"num_write":1177,"num_write_kb":31344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":3372198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":138,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.949037+0000","last_change":"2024-09-12T09:33:51.949037+0000","last_active":"2024-09-12T09:33:51.949037+0000","last_peered":"2024-09-12T09:33:51.949037+0000","last_clean":"2024-09-12T09:33:51.949037+0000","last_became_active":"2024-09-12T09:33:38.793051+0000","last_became_peered":"2024-09-12T09:33:38.793051+0000","last_unstale":"2024-09-12T09:33:51.949037+0000","last_undegraded":"2024-09-12T09:33:51.949037+0000","last_fullsized":"2024-09-12T09:33:51.949037+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:51.948843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:51.948843+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.948843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:43.318237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"236'424","reported_seq":7372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.488430+0000","last_change":"2024-09-12T09:33:50.488430+0000","last_active":"2024-09-12T09:33:50.488430+0000","last_peered":"2024-09-12T09:33:50.488430+0000","last_clean":"2024-09-12T09:33:50.488430+0000","last_became_active":"2024-09-12T09:33:25.717744+0000","last_became_peered":"2024-09-12T09:33:25.717744+0000","last_unstale":"2024-09-12T09:33:50.488430+0000","last_undegraded":"2024-09-12T09:33:50.488430+0000","last_fullsized":"2024-09-12T09:33:50.488430+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'424","last_scrub_stamp":"2024-09-12T09:33:50.488356+0000","last_deep_scrub":"236'424","last_deep_scrub_stamp":"2024-09-12T09:33:50.488356+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.488356+0000","objects_scrubbed":5,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:59.989965+0000","scrub_duration":3,"objects_trimmed":30,"snaptrim_duration":0.12921821999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6905,"num_read_kb":29670,"num_write":317,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":25569246,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":241,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.246791+0000","last_change":"2024-09-12T09:33:51.246791+0000","last_active":"2024-09-12T09:33:51.246791+0000","last_peered":"2024-09-12T09:33:51.246791+0000","last_clean":"2024-09-12T09:33:51.246791+0000","last_became_active":"2024-09-12T09:31:39.763141+0000","last_became_peered":"2024-09-12T09:31:39.763141+0000","last_unstale":"2024-09-12T09:33:51.246791+0000","last_undegraded":"2024-09-12T09:33:51.246791+0000","last_fullsized":"2024-09-12T09:33:51.246791+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:51.246723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:51.246723+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.246723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:59:19.630960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001199706,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"236'449","reported_seq":4210,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.749691+0000","last_change":"2024-09-12T09:33:50.749691+0000","last_active":"2024-09-12T09:33:50.749691+0000","last_peered":"2024-09-12T09:33:50.749691+0000","last_clean":"2024-09-12T09:33:50.749691+0000","last_became_active":"2024-09-12T09:33:32.789363+0000","last_became_peered":"2024-09-12T09:33:32.789363+0000","last_unstale":"2024-09-12T09:33:50.749691+0000","last_undegraded":"2024-09-12T09:33:50.749691+0000","last_fullsized":"2024-09-12T09:33:50.749691+0000","mapping_epoch":226,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":227,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'449","last_scrub_stamp":"2024-09-12T09:33:50.749572+0000","last_deep_scrub":"236'449","last_deep_scrub_stamp":"2024-09-12T09:33:50.749572+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.749572+0000","objects_scrubbed":7,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:56.817190+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0049398030000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3932,"num_read_kb":30622,"num_write":467,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4239133,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.360079+0000","last_change":"2024-09-12T09:33:53.360079+0000","last_active":"2024-09-12T09:33:53.360079+0000","last_peered":"2024-09-12T09:33:53.360079+0000","last_clean":"2024-09-12T09:33:53.360079+0000","last_became_active":"2024-09-12T09:33:22.713651+0000","last_became_peered":"2024-09-12T09:33:22.713651+0000","last_unstale":"2024-09-12T09:33:53.360079+0000","last_undegraded":"2024-09-12T09:33:53.360079+0000","last_fullsized":"2024-09-12T09:33:53.360079+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:53.359895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:53.359895+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.359895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:24.669756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098178099999999993,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"236'1147","reported_seq":8235,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.297929+0000","last_change":"2024-09-12T09:33:53.297929+0000","last_active":"2024-09-12T09:33:53.297929+0000","last_peered":"2024-09-12T09:33:53.297929+0000","last_clean":"2024-09-12T09:33:53.297929+0000","last_became_active":"2024-09-12T09:31:43.681130+0000","last_became_peered":"2024-09-12T09:31:43.681130+0000","last_unstale":"2024-09-12T09:33:53.297929+0000","last_undegraded":"2024-09-12T09:33:53.297929+0000","last_fullsized":"2024-09-12T09:33:53.297929+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'1147","last_scrub_stamp":"2024-09-12T09:33:53.297748+0000","last_deep_scrub":"236'1147","last_deep_scrub_stamp":"2024-09-12T09:33:53.297748+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.297748+0000","objects_scrubbed":2,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:18:44.020746+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0072538339999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11145,"num_read_kb":45471,"num_write":1748,"num_write_kb":50822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":17646785,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":256,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.248053+0000","last_change":"2024-09-12T09:33:50.248053+0000","last_active":"2024-09-12T09:33:50.248053+0000","last_peered":"2024-09-12T09:33:50.248053+0000","last_clean":"2024-09-12T09:33:50.248053+0000","last_became_active":"2024-09-12T09:33:22.713568+0000","last_became_peered":"2024-09-12T09:33:22.713568+0000","last_unstale":"2024-09-12T09:33:50.248053+0000","last_undegraded":"2024-09-12T09:33:50.248053+0000","last_fullsized":"2024-09-12T09:33:50.248053+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:50.247987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:50.247987+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.247987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:10:25.406300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001155671,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"236'312","reported_seq":14365,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.939403+0000","last_change":"2024-09-12T09:33:52.939403+0000","last_active":"2024-09-12T09:33:52.939403+0000","last_peered":"2024-09-12T09:33:52.939403+0000","last_clean":"2024-09-12T09:33:52.939403+0000","last_became_active":"2024-09-12T09:29:34.668291+0000","last_became_peered":"2024-09-12T09:29:34.668291+0000","last_unstale":"2024-09-12T09:33:52.939403+0000","last_undegraded":"2024-09-12T09:33:52.939403+0000","last_fullsized":"2024-09-12T09:33:52.939403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'312","last_scrub_stamp":"2024-09-12T09:33:52.939164+0000","last_deep_scrub":"236'312","last_deep_scrub_stamp":"2024-09-12T09:33:52.939164+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.939164+0000","objects_scrubbed":16,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:08.476417+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0096059200000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13731,"num_read_kb":60682,"num_write":239,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"236'1229","reported_seq":6838,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.861514+0000","last_change":"2024-09-12T09:33:51.861514+0000","last_active":"2024-09-12T09:33:51.861514+0000","last_peered":"2024-09-12T09:33:51.861514+0000","last_clean":"2024-09-12T09:33:51.861514+0000","last_became_active":"2024-09-12T09:33:25.714008+0000","last_became_peered":"2024-09-12T09:33:25.714008+0000","last_unstale":"2024-09-12T09:33:51.861514+0000","last_undegraded":"2024-09-12T09:33:51.861514+0000","last_fullsized":"2024-09-12T09:33:51.861514+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'1229","last_scrub_stamp":"2024-09-12T09:33:51.861290+0000","last_deep_scrub":"236'1229","last_deep_scrub_stamp":"2024-09-12T09:33:51.861290+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.861290+0000","objects_scrubbed":15,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:57:31.999477+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0025995990000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8232,"num_read_kb":25472,"num_write":1563,"num_write_kb":11366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":8604756,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"236'3","reported_seq":500,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.904514+0000","last_change":"2024-09-12T09:33:51.904514+0000","last_active":"2024-09-12T09:33:51.904514+0000","last_peered":"2024-09-12T09:33:51.904514+0000","last_clean":"2024-09-12T09:33:51.904514+0000","last_became_active":"2024-09-12T09:31:39.763139+0000","last_became_peered":"2024-09-12T09:31:39.763139+0000","last_unstale":"2024-09-12T09:33:51.904514+0000","last_undegraded":"2024-09-12T09:33:51.904514+0000","last_fullsized":"2024-09-12T09:33:51.904514+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'3","last_scrub_stamp":"2024-09-12T09:33:51.904330+0000","last_deep_scrub":"236'3","last_deep_scrub_stamp":"2024-09-12T09:33:51.904330+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.904330+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:40:22.815837+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021491430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"236'426","reported_seq":5287,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.305376+0000","last_change":"2024-09-12T09:33:52.305376+0000","last_active":"2024-09-12T09:33:52.305376+0000","last_peered":"2024-09-12T09:33:52.305376+0000","last_clean":"2024-09-12T09:33:52.305376+0000","last_became_active":"2024-09-12T09:31:44.676172+0000","last_became_peered":"2024-09-12T09:31:44.676172+0000","last_unstale":"2024-09-12T09:33:52.305376+0000","last_undegraded":"2024-09-12T09:33:52.305376+0000","last_fullsized":"2024-09-12T09:33:52.305376+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'426","last_scrub_stamp":"2024-09-12T09:33:52.305171+0000","last_deep_scrub":"236'426","last_deep_scrub_stamp":"2024-09-12T09:33:52.305171+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.305171+0000","objects_scrubbed":12,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:49.815798+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.006654615,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4690,"num_read_kb":38017,"num_write":349,"num_write_kb":16436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":23827542,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":460,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.755055+0000","last_change":"2024-09-12T09:33:50.755055+0000","last_active":"2024-09-12T09:33:50.755055+0000","last_peered":"2024-09-12T09:33:50.755055+0000","last_clean":"2024-09-12T09:33:50.755055+0000","last_became_active":"2024-09-12T09:29:24.330210+0000","last_became_peered":"2024-09-12T09:29:24.330210+0000","last_unstale":"2024-09-12T09:33:50.755055+0000","last_undegraded":"2024-09-12T09:33:50.755055+0000","last_fullsized":"2024-09-12T09:33:50.755055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:50.754913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:50.754913+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.754913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:21:58.495614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097830599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"236'538","reported_seq":13578,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.722456+0000","last_change":"2024-09-12T09:33:51.722456+0000","last_active":"2024-09-12T09:33:51.722456+0000","last_peered":"2024-09-12T09:33:51.722456+0000","last_clean":"2024-09-12T09:33:51.722456+0000","last_became_active":"2024-09-12T09:32:52.774699+0000","last_became_peered":"2024-09-12T09:32:52.774699+0000","last_unstale":"2024-09-12T09:33:51.722456+0000","last_undegraded":"2024-09-12T09:33:51.722456+0000","last_fullsized":"2024-09-12T09:33:51.722456+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'538","last_scrub_stamp":"2024-09-12T09:33:51.722392+0000","last_deep_scrub":"236'538","last_deep_scrub_stamp":"2024-09-12T09:33:51.722392+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.722392+0000","objects_scrubbed":6,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:29.340434+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0010988250000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14234,"num_read_kb":39342,"num_write":545,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":15932247,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":369,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:54.381471+0000","last_change":"2024-09-12T09:33:54.381471+0000","last_active":"2024-09-12T09:33:54.381471+0000","last_peered":"2024-09-12T09:33:54.381471+0000","last_clean":"2024-09-12T09:33:54.381471+0000","last_became_active":"2024-09-12T09:31:39.763202+0000","last_became_peered":"2024-09-12T09:31:39.763202+0000","last_unstale":"2024-09-12T09:33:54.381471+0000","last_undegraded":"2024-09-12T09:33:54.381471+0000","last_fullsized":"2024-09-12T09:33:54.381471+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T09:33:54.381271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T09:33:54.381271+0000","last_clean_scrub_stamp":"2024-09-12T09:33:54.381271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:22.012429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"236'427","reported_seq":6046,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.494599+0000","last_change":"2024-09-12T09:33:52.494599+0000","last_active":"2024-09-12T09:33:52.494599+0000","last_peered":"2024-09-12T09:33:52.494599+0000","last_clean":"2024-09-12T09:33:52.494599+0000","last_became_active":"2024-09-12T09:33:26.733945+0000","last_became_peered":"2024-09-12T09:33:26.733945+0000","last_unstale":"2024-09-12T09:33:52.494599+0000","last_undegraded":"2024-09-12T09:33:52.494599+0000","last_fullsized":"2024-09-12T09:33:52.494599+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'427","last_scrub_stamp":"2024-09-12T09:33:52.494411+0000","last_deep_scrub":"236'427","last_deep_scrub_stamp":"2024-09-12T09:33:52.494411+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.494411+0000","objects_scrubbed":10,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:44.511993+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0030836829999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6486,"num_read_kb":47168,"num_write":395,"num_write_kb":14941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":12202442,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"236'2","reported_seq":459,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.965581+0000","last_change":"2024-09-12T09:33:53.965581+0000","last_active":"2024-09-12T09:33:53.965581+0000","last_peered":"2024-09-12T09:33:53.965581+0000","last_clean":"2024-09-12T09:33:53.965581+0000","last_became_active":"2024-09-12T09:29:23.942767+0000","last_became_peered":"2024-09-12T09:29:23.942767+0000","last_unstale":"2024-09-12T09:33:53.965581+0000","last_undegraded":"2024-09-12T09:33:53.965581+0000","last_fullsized":"2024-09-12T09:33:53.965581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'2","last_scrub_stamp":"2024-09-12T09:33:53.965390+0000","last_deep_scrub":"236'2","last_deep_scrub_stamp":"2024-09-12T09:33:53.965390+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.965390+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:50:23.741310+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0011869999999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"236'74","reported_seq":404,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.383838+0000","last_change":"2024-09-12T09:33:52.383838+0000","last_active":"2024-09-12T09:33:52.383838+0000","last_peered":"2024-09-12T09:33:52.383838+0000","last_clean":"2024-09-12T09:33:52.383838+0000","last_became_active":"2024-09-12T09:33:22.724758+0000","last_became_peered":"2024-09-12T09:33:22.724758+0000","last_unstale":"2024-09-12T09:33:52.383838+0000","last_undegraded":"2024-09-12T09:33:52.383838+0000","last_fullsized":"2024-09-12T09:33:52.383838+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'74","last_scrub_stamp":"2024-09-12T09:33:52.383648+0000","last_deep_scrub":"236'74","last_deep_scrub_stamp":"2024-09-12T09:33:52.383648+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.383648+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:35:58.456292+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"236'1334","reported_seq":6625,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:51.748688+0000","last_change":"2024-09-12T09:33:51.748599+0000","last_active":"2024-09-12T09:33:51.748688+0000","last_peered":"2024-09-12T09:33:51.748688+0000","last_clean":"2024-09-12T09:33:51.748688+0000","last_became_active":"2024-09-12T09:31:43.682501+0000","last_became_peered":"2024-09-12T09:31:43.682501+0000","last_unstale":"2024-09-12T09:33:51.748688+0000","last_undegraded":"2024-09-12T09:33:51.748688+0000","last_fullsized":"2024-09-12T09:33:51.748688+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'1334","last_scrub_stamp":"2024-09-12T09:33:51.748471+0000","last_deep_scrub":"236'1334","last_deep_scrub_stamp":"2024-09-12T09:33:51.748471+0000","last_clean_scrub_stamp":"2024-09-12T09:33:51.748471+0000","objects_scrubbed":15,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:10:37.755106+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0026190879999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":11440,"num_read_kb":57223,"num_write":1942,"num_write_kb":24778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":23016330,"num_keys_recovered":54,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"236'741","reported_seq":5193,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:50.851603+0000","last_change":"2024-09-12T09:33:50.851466+0000","last_active":"2024-09-12T09:33:50.851603+0000","last_peered":"2024-09-12T09:33:50.851603+0000","last_clean":"2024-09-12T09:33:50.851603+0000","last_became_active":"2024-09-12T09:29:34.656523+0000","last_became_peered":"2024-09-12T09:29:34.656523+0000","last_unstale":"2024-09-12T09:33:50.851603+0000","last_undegraded":"2024-09-12T09:33:50.851603+0000","last_fullsized":"2024-09-12T09:33:50.851603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'741","last_scrub_stamp":"2024-09-12T09:33:50.851248+0000","last_deep_scrub":"236'741","last_deep_scrub_stamp":"2024-09-12T09:33:50.851248+0000","last_clean_scrub_stamp":"2024-09-12T09:33:50.851248+0000","objects_scrubbed":14,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:57.416130+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0024055040000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6054,"num_read_kb":30020,"num_write":919,"num_write_kb":15066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"236'516","reported_seq":6026,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:53.982736+0000","last_change":"2024-09-12T09:33:53.982736+0000","last_active":"2024-09-12T09:33:53.982736+0000","last_peered":"2024-09-12T09:33:53.982736+0000","last_clean":"2024-09-12T09:33:53.982736+0000","last_became_active":"2024-09-12T09:33:38.712992+0000","last_became_peered":"2024-09-12T09:33:38.712992+0000","last_unstale":"2024-09-12T09:33:53.982736+0000","last_undegraded":"2024-09-12T09:33:53.982736+0000","last_fullsized":"2024-09-12T09:33:53.982736+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'516","last_scrub_stamp":"2024-09-12T09:33:53.982542+0000","last_deep_scrub":"236'516","last_deep_scrub_stamp":"2024-09-12T09:33:53.982542+0000","last_clean_scrub_stamp":"2024-09-12T09:33:53.982542+0000","objects_scrubbed":10,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:20:53.516249+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.002183731,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5229,"num_read_kb":17441,"num_write":561,"num_write_kb":10447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":12960978,"num_keys_recovered":89,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"236'522","reported_seq":7096,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.769094+0000","last_change":"2024-09-12T09:33:52.768950+0000","last_active":"2024-09-12T09:33:52.769094+0000","last_peered":"2024-09-12T09:33:52.769094+0000","last_clean":"2024-09-12T09:33:52.769094+0000","last_became_active":"2024-09-12T09:33:25.713779+0000","last_became_peered":"2024-09-12T09:33:25.713779+0000","last_unstale":"2024-09-12T09:33:52.769094+0000","last_undegraded":"2024-09-12T09:33:52.769094+0000","last_fullsized":"2024-09-12T09:33:52.769094+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'522","last_scrub_stamp":"2024-09-12T09:33:52.768744+0000","last_deep_scrub":"236'522","last_deep_scrub_stamp":"2024-09-12T09:33:52.768744+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.768744+0000","objects_scrubbed":9,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:59:06.079659+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0036212340000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6746,"num_read_kb":72342,"num_write":648,"num_write_kb":32943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":31700540,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"236'518","reported_seq":4660,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:54.933577+0000","last_change":"2024-09-12T09:33:54.933437+0000","last_active":"2024-09-12T09:33:54.933577+0000","last_peered":"2024-09-12T09:33:54.933577+0000","last_clean":"2024-09-12T09:33:54.933577+0000","last_became_active":"2024-09-12T09:29:34.668181+0000","last_became_peered":"2024-09-12T09:29:34.668181+0000","last_unstale":"2024-09-12T09:33:54.933577+0000","last_undegraded":"2024-09-12T09:33:54.933577+0000","last_fullsized":"2024-09-12T09:33:54.933577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'518","last_scrub_stamp":"2024-09-12T09:33:54.933216+0000","last_deep_scrub":"236'518","last_deep_scrub_stamp":"2024-09-12T09:33:54.933216+0000","last_clean_scrub_stamp":"2024-09-12T09:33:54.933216+0000","objects_scrubbed":7,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:15:19.505557+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0062650630000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5763,"num_read_kb":44866,"num_write":464,"num_write_kb":19917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"236'609","reported_seq":9252,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:55.426450+0000","last_change":"2024-09-12T09:33:55.426450+0000","last_active":"2024-09-12T09:33:55.426450+0000","last_peered":"2024-09-12T09:33:55.426450+0000","last_clean":"2024-09-12T09:33:55.426450+0000","last_became_active":"2024-09-12T09:30:33.675043+0000","last_became_peered":"2024-09-12T09:30:33.675043+0000","last_unstale":"2024-09-12T09:33:55.426450+0000","last_undegraded":"2024-09-12T09:33:55.426450+0000","last_fullsized":"2024-09-12T09:33:55.426450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'609","last_scrub_stamp":"2024-09-12T09:33:55.426215+0000","last_deep_scrub":"236'609","last_deep_scrub_stamp":"2024-09-12T09:33:55.426215+0000","last_clean_scrub_stamp":"2024-09-12T09:33:55.426215+0000","objects_scrubbed":12,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.522044+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0017243060000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":9714,"num_read_kb":29065,"num_write":613,"num_write_kb":7809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2175621,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"236'638","reported_seq":6531,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:55.899016+0000","last_change":"2024-09-12T09:33:55.898877+0000","last_active":"2024-09-12T09:33:55.899016+0000","last_peered":"2024-09-12T09:33:55.899016+0000","last_clean":"2024-09-12T09:33:55.899016+0000","last_became_active":"2024-09-12T09:33:26.735617+0000","last_became_peered":"2024-09-12T09:33:26.735617+0000","last_unstale":"2024-09-12T09:33:55.899016+0000","last_undegraded":"2024-09-12T09:33:55.899016+0000","last_fullsized":"2024-09-12T09:33:55.899016+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'638","last_scrub_stamp":"2024-09-12T09:33:55.898689+0000","last_deep_scrub":"236'638","last_deep_scrub_stamp":"2024-09-12T09:33:55.898689+0000","last_clean_scrub_stamp":"2024-09-12T09:33:55.898689+0000","objects_scrubbed":12,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:53:24.931128+0000","scrub_duration":19,"objects_trimmed":32,"snaptrim_duration":0.209963128,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7345,"num_read_kb":94629,"num_write":660,"num_write_kb":45700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":119954602,"num_keys_recovered":83,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"236'707","reported_seq":7655,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:57.496451+0000","last_change":"2024-09-12T09:33:57.496451+0000","last_active":"2024-09-12T09:33:57.496451+0000","last_peered":"2024-09-12T09:33:57.496451+0000","last_clean":"2024-09-12T09:33:57.496451+0000","last_became_active":"2024-09-12T09:31:12.221090+0000","last_became_peered":"2024-09-12T09:31:12.221090+0000","last_unstale":"2024-09-12T09:33:57.496451+0000","last_undegraded":"2024-09-12T09:33:57.496451+0000","last_fullsized":"2024-09-12T09:33:57.496451+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'707","last_scrub_stamp":"2024-09-12T09:33:57.496265+0000","last_deep_scrub":"236'707","last_deep_scrub_stamp":"2024-09-12T09:33:57.496265+0000","last_clean_scrub_stamp":"2024-09-12T09:33:57.496265+0000","objects_scrubbed":8,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:04.516946+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001442289,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":7603,"num_read_kb":80217,"num_write":766,"num_write_kb":46032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":109851818,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"236'695","reported_seq":4319,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:52.999031+0000","last_change":"2024-09-12T09:33:52.999031+0000","last_active":"2024-09-12T09:33:52.999031+0000","last_peered":"2024-09-12T09:33:52.999031+0000","last_clean":"2024-09-12T09:33:52.999031+0000","last_became_active":"2024-09-12T09:33:38.714673+0000","last_became_peered":"2024-09-12T09:33:38.714673+0000","last_unstale":"2024-09-12T09:33:52.999031+0000","last_undegraded":"2024-09-12T09:33:52.999031+0000","last_fullsized":"2024-09-12T09:33:52.999031+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'695","last_scrub_stamp":"2024-09-12T09:33:52.998795+0000","last_deep_scrub":"236'695","last_deep_scrub_stamp":"2024-09-12T09:33:52.998795+0000","last_clean_scrub_stamp":"2024-09-12T09:33:52.998795+0000","objects_scrubbed":14,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:34:57.870642+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.00078837099999999995,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3710,"num_read_kb":27724,"num_write":823,"num_write_kb":26761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":26271537,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"236'681","reported_seq":3732,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:56.465659+0000","last_change":"2024-09-12T09:33:56.465659+0000","last_active":"2024-09-12T09:33:56.465659+0000","last_peered":"2024-09-12T09:33:56.465659+0000","last_clean":"2024-09-12T09:33:56.465659+0000","last_became_active":"2024-09-12T09:30:30.874342+0000","last_became_peered":"2024-09-12T09:30:30.874342+0000","last_unstale":"2024-09-12T09:33:56.465659+0000","last_undegraded":"2024-09-12T09:33:56.465659+0000","last_fullsized":"2024-09-12T09:33:56.465659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'681","last_scrub_stamp":"2024-09-12T09:33:56.465430+0000","last_deep_scrub":"236'681","last_deep_scrub_stamp":"2024-09-12T09:33:56.465430+0000","last_clean_scrub_stamp":"2024-09-12T09:33:56.465430+0000","objects_scrubbed":10,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:49:22.146521+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0013825689999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5647,"num_read_kb":20018,"num_write":773,"num_write_kb":9436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":906303,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"236'483","reported_seq":13958,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:33:58.463977+0000","last_change":"2024-09-12T09:33:58.463977+0000","last_active":"2024-09-12T09:33:58.463977+0000","last_peered":"2024-09-12T09:33:58.463977+0000","last_clean":"2024-09-12T09:33:58.463977+0000","last_became_active":"2024-09-12T09:32:45.711157+0000","last_became_peered":"2024-09-12T09:32:45.711157+0000","last_unstale":"2024-09-12T09:33:58.463977+0000","last_undegraded":"2024-09-12T09:33:58.463977+0000","last_fullsized":"2024-09-12T09:33:58.463977+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'483","last_scrub_stamp":"2024-09-12T09:33:58.463857+0000","last_deep_scrub":"236'483","last_deep_scrub_stamp":"2024-09-12T09:33:58.463857+0000","last_clean_scrub_stamp":"2024-09-12T09:33:58.463857+0000","objects_scrubbed":13,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:17.041141+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0010313989999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13606,"num_read_kb":130085,"num_write":540,"num_write_kb":65722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":228417119,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"236'265","reported_seq":2070,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-12T09:34:00.415260+0000","last_change":"2024-09-12T09:34:00.415260+0000","last_active":"2024-09-12T09:34:00.415260+0000","last_peered":"2024-09-12T09:34:00.415260+0000","last_clean":"2024-09-12T09:34:00.415260+0000","last_became_active":"2024-09-12T09:30:30.878243+0000","last_became_peered":"2024-09-12T09:30:30.878243+0000","last_unstale":"2024-09-12T09:34:00.415260+0000","last_undegraded":"2024-09-12T09:34:00.415260+0000","last_fullsized":"2024-09-12T09:34:00.415260+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"236'265","last_scrub_stamp":"2024-09-12T09:34:00.415066+0000","last_deep_scrub":"236'265","last_deep_scrub_stamp":"2024-09-12T09:34:00.415066+0000","last_clean_scrub_stamp":"2024-09-12T09:34:00.415066+0000","objects_scrubbed":6,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:47:29.993740+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001777659,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1627,"num_read_kb":9782,"num_write":321,"num_write_kb":13505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":5713687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":318,"num_object_clones":0,"num_object_copies":636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":314,"num_read":256433,"num_read_kb":1583705,"num_write":23310,"num_write_kb":852605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5183,"num_bytes_recovered":841343925,"num_keys_recovered":727,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},"log_size":20429,"ondisk_log_size":20429,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3542208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":2951840,"data_compressed":33270,"data_compressed_allocated":184320,"data_compressed_original":2949120,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64536,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307304,"statfs":{"total":96636764160,"available":96570679296,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52020,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319820,"statfs":{"total":96636764160,"available":96583495680,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29808,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342032,"statfs":{"total":96636764160,"available":96606240768,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2788,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2854912,"data_stored":1408968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542209,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56176,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315664,"statfs":{"total":96636764160,"available":96579239936,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542209,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60336,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311504,"statfs":{"total":96636764160,"available":96574980096,"internally_reserved":0,"allocated":2850816,"data_stored":1408949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542207,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33976,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337864,"statfs":{"total":96636764160,"available":96601972736,"internally_reserved":0,"allocated":2859008,"data_stored":1408987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":216,"seq":927712935948,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36184,"kb_used_data":2824,"kb_used_omap":40,"kb_used_meta":33303,"kb_avail":94335656,"statfs":{"total":96636764160,"available":96599711744,"internally_reserved":0,"allocated":2891776,"data_stored":1999317,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":41759,"internal_metadata":34102497},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6654,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16027,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T09:34:12.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-12T09:34:13.233 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T09:34:13.233 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T09:34:13.233 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T09:34:13.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:13.288 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T09:34:13.288 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-12T09:34:13.288 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:18.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-12T09:34:19.390 INFO:tasks.ceph.osd.1:Stopped 2024-09-12T09:34:19.390 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T09:34:19.391 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:24.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-12T09:34:24.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-12T09:34:25.493 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T09:34:25.493 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-12T09:34:25.493 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:25.593 INFO:tasks.ceph.osd.3:Stopped 2024-09-12T09:34:25.594 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-12T09:34:25.594 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:30.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-12T09:34:30.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-12T09:34:31.696 INFO:tasks.ceph.osd.4:Stopped 2024-09-12T09:34:31.696 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-12T09:34:31.696 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:31.766 INFO:tasks.ceph.osd.5:Stopped 2024-09-12T09:34:31.767 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-12T09:34:31.767 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:36.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-12T09:34:36.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-12T09:34:37.868 INFO:tasks.ceph.osd.6:Stopped 2024-09-12T09:34:37.869 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-12T09:34:37.869 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:37.941 INFO:tasks.ceph.osd.7:Stopped 2024-09-12T09:34:37.942 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T09:34:37.942 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-12T09:34:37.942 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:37.977 INFO:tasks.ceph.mgr.y:Stopped 2024-09-12T09:34:37.977 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-12T09:34:37.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:38.008 INFO:tasks.ceph.mgr.x:Stopped 2024-09-12T09:34:38.009 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T09:34:38.009 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-12T09:34:38.009 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:38.033 INFO:tasks.ceph.mon.a:Stopped 2024-09-12T09:34:38.033 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-12T09:34:38.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:38.089 INFO:tasks.ceph.mon.c:Stopped 2024-09-12T09:34:38.090 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-12T09:34:38.090 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T09:34:38.104 INFO:tasks.ceph.mon.b:Stopped 2024-09-12T09:34:38.105 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T09:34:38.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-12T09:34:38.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:34:38.148 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-12T09:34:38.350 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:34:38.350 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-12T09:34:38.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:34:38.535 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-12T09:34:38.678 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:34:38.679 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-12T09:34:38.843 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:34:38.843 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-12T09:34:39.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:34:39.147 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-12T09:34:39.318 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:34:39.319 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-12T09:34:39.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:34:39.510 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-12T09:34:39.701 INFO:tasks.ceph:Archiving mon data... 2024-09-12T09:34:39.702 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/data/mon.a.tgz 2024-09-12T09:34:39.703 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-12T09:34:39.709 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.SDKjs5tN9v 2024-09-12T09:34:39.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SDKjs5tN9v 2024-09-12T09:34:39.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.SDKjs5tN9v 2024-09-12T09:34:39.944 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.SDKjs5tN9v is 516KB 2024-09-12T09:34:40.005 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.SDKjs5tN9v 2024-09-12T09:34:40.010 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/data/mon.c.tgz 2024-09-12T09:34:40.010 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-12T09:34:40.055 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.gQUsxGphXY 2024-09-12T09:34:40.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gQUsxGphXY 2024-09-12T09:34:40.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.gQUsxGphXY 2024-09-12T09:34:40.334 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.gQUsxGphXY is 700KB 2024-09-12T09:34:40.397 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.gQUsxGphXY 2024-09-12T09:34:40.402 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/data/mon.b.tgz 2024-09-12T09:34:40.402 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-12T09:34:40.408 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.oLFGJxnUkt 2024-09-12T09:34:40.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.oLFGJxnUkt 2024-09-12T09:34:40.621 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.oLFGJxnUkt 2024-09-12T09:34:40.686 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.oLFGJxnUkt is 700KB 2024-09-12T09:34:40.749 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.oLFGJxnUkt 2024-09-12T09:34:40.754 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T09:34:40.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T09:34:40.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T09:34:40.815 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T09:34:40.856 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T09:34:40.856 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T09:34:40.857 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi007/crash 2024-09-12T09:34:40.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T09:34:40.873 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi115/crash 2024-09-12T09:34:40.874 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T09:34:40.911 INFO:tasks.ceph:Compressing logs... 2024-09-12T09:34:40.911 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T09:34:40.924 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T09:34:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-09-12T09:34:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33304.log 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30620.log 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43958.log: /var/log/ceph/ceph-client.admin.33304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33304.log.gz 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30620.log.gz 2024-09-12T09:34:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32128.log 2024-09-12T09:34:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45128.log 2024-09-12T09:34:40.953 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-09-12T09:34:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32128.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38815.log 2024-09-12T09:34:40.953 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32128.log.gz 2024-09-12T09:34:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45128.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45128.log.gz -5 2024-09-12T09:34:40.953 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32477.log 2024-09-12T09:34:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32344.log 2024-09-12T09:34:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.38815.log: /var/log/ceph/ceph-client.admin.32477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35939.log 2024-09-12T09:34:40.954 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32477.log.gz 2024-09-12T09:34:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35466.log 2024-09-12T09:34:40.954 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32344.log.gz 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30134.log 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35939.log.gz 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35466.log.gz 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33631.log 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30134.log.gz 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-09-12T09:34:40.957 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31249.log 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33631.log.gz 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42543.log 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31249.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31249.log.gz -5 2024-09-12T09:34:40.958 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30037.log 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35719.log 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42543.log.gz 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30037.log.gz 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35719.log.gz 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42263.log 2024-09-12T09:34:40.964 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37272.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44376.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42263.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44376.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.38815.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40229.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44376.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42060.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40229.log.gz 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35678.log 2024-09-12T09:34:40.965 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34694.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42060.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33578.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35678.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.34694.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30160.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34694.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32742.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33578.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30160.log.gz 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33209.log 2024-09-12T09:34:40.966 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32742.log.gz 2024-09-12T09:34:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45336.log 2024-09-12T09:34:40.972 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44575.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33209.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45336.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44575.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44575.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42909.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30966.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36462.log 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42909.log.gz 2024-09-12T09:34:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34027.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30966.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45284.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36462.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31037.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34027.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40080.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45284.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.31037.log: -- /var/log/ceph/ceph-client.admin.36595.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31037.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33413.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40080.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32272.log 2024-09-12T09:34:40.974 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33413.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42140.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32272.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42140.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44368.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34913.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37338.log 2024-09-12T09:34:40.975 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44368.log.gz 2024-09-12T09:34:40.991 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26329.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26377.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26329.log.gz 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26377.log.gz 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26504.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26401.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26504.log.gz 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26449.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26401.log.gz 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26353.log 2024-09-12T09:34:40.992 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26449.log.gz 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26425.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26353.log.gz 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26425.log.gz 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26473.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.26473.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26473.log.gz 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.b.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T09:34:40.993 INFO:teuthology.orchestra.run.smithi115.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30212.log 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34913.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34476.log 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37338.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45492.log 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30212.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34476.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45596.log 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45492.log.gz 2024-09-12T09:34:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40225.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45596.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37595.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr: 25.5% -- replaced with /var/log/ceph/ceph-client.admin.40225.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30996.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30082.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37595.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30996.log.gz 2024-09-12T09:34:41.011 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32962.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30082.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33989.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34939.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32962.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44784.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33989.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35309.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34939.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44784.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37060.log 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35309.log.gz 2024-09-12T09:34:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31726.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32841.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37060.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31726.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37034.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32841.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35182.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40806.log 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37034.log.gz 2024-09-12T09:34:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45024.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35182.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.40806.log: --verbose -- /var/log/ceph/ceph-client.admin.32033.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40806.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45024.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.32033.log: --verbose -- /var/log/ceph/ceph-client.admin.41920.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32033.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37488.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41920.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38422.log 2024-09-12T09:34:41.014 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37488.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34392.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44972.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38422.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34392.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34392.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44972.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-09-12T09:34:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37008.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40201.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37008.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30888.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gzgzip 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36668.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35864.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30888.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36668.log.gz 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34504.log 2024-09-12T09:34:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35864.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34504.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35494.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35494.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33627.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45206.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33627.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41600.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30862.log 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41600.log.gz 2024-09-12T09:34:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39787.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30862.log: /var/log/ceph/ceph-client.admin.44062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30862.log.gz 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40705.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39787.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39787.log.gz -5 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42013.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33108.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40705.log: 0.0%/var/log/ceph/ceph-client.admin.42013.log: -- replaced with /var/log/ceph/ceph-client.admin.40705.log.gz 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42013.log.gz 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31347.log 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33108.log.gz 2024-09-12T09:34:41.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31347.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33049.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44583.log: /var/log/ceph/ceph-client.admin.33049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33049.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36967.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45180.log: /var/log/ceph/ceph-client.admin.36967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36967.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30621.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30630.log 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45388.log: /var/log/ceph/ceph-client.admin.30621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-09-12T09:34:41.019 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30621.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34983.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34098.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30630.log: /var/log/ceph/ceph-client.admin.34983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30630.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34983.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30012.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31972.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34098.log: /var/log/ceph/ceph-client.admin.30012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34098.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30012.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39735.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41802.log 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31972.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39735.log.gz 2024-09-12T09:34:41.020 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35120.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41802.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35120.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35120.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39979.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34887.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39979.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36913.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30409.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34887.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36913.log.gz 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31398.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30412.log 2024-09-12T09:34:41.021 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30409.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31398.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41806.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41395.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30412.log.gz/var/log/ceph/ceph-client.admin.41806.log: 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41806.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33764.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30414.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41395.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33764.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41505.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41063.log 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30414.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41505.log.gz 2024-09-12T09:34:41.022 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40481.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32518.log/var/log/ceph/ceph-client.admin.41063.log: 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41063.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40481.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32079.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30403.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32518.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32079.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35745.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30403.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32556.log 2024-09-12T09:34:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35745.log.gz 2024-09-12T09:34:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-09-12T09:34:41.024 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43906.log 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32556.log.gz 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31549.log 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42478.log 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43906.log.gz 2024-09-12T09:34:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31549.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31549.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42478.log: /var/log/ceph/ceph-client.admin.45414.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42478.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36232.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44361.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45414.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39940.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36232.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44361.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44780.log 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39940.log.gz 2024-09-12T09:34:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39876.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36030.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44780.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26242.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39876.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42430.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36030.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.26242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 28.4% -- replaced with /var/log/ceph/ceph-client.admin.26242.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44192.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42430.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-09-12T09:34:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40604.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44192.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30062.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40604.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42884.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31554.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30062.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45310.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42884.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31554.log.gz 2024-09-12T09:34:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42404.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45310.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30605.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35095.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42404.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30914.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30605.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44774.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35095.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39424.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30914.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42933.log 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44774.log.gz 2024-09-12T09:34:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39424.log.gz 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42476.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42933.log.gz 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32182.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32880.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42476.log.gz 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35571.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32182.log.gz 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-09-12T09:34:41.058 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32880.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41700.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35571.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35276.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41700.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30622.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35359.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35276.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36231.log 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30622.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35359.log.gz 2024-09-12T09:34:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36231.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36231.log.gz -- 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.33888.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30404.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31660.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33888.log: 0.0%/var/log/ceph/ceph-client.admin.30404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33888.log.gz 0.0% 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30404.log.gz 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31660.log.gz 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45570.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36784.log 2024-09-12T09:34:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34251.log 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gz 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33369.log 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36784.log: /var/log/ceph/ceph-client.admin.34251.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.32397.log -- replaced with /var/log/ceph/ceph-client.admin.36784.log.gz 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34251.log.gz 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33369.log.gz 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31204.log 2024-09-12T09:34:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35976.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32397.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30108.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31204.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39694.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35976.log: /var/log/ceph/ceph-client.admin.30108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41194.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35976.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30108.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32683.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39694.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39646.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41194.log: /var/log/ceph/ceph-client.admin.32683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45102.log 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41194.log.gz 2024-09-12T09:34:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32683.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39646.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30606.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45102.log: /var/log/ceph/ceph-client.admin.30606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32658.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45102.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30606.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41428.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29895.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34190.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32658.log: /var/log/ceph/ceph-client.admin.29895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30810.log 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32658.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29895.log.gz 2024-09-12T09:34:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34190.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.34190.log.gz --verbose 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.40518.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31036.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30810.log: /var/log/ceph/ceph-client.admin.40518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36466.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30810.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40518.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31036.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.31036.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44782.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45050.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36466.log: /var/log/ceph/ceph-client.admin.44782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37567.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36466.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44782.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45050.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45050.log.gz 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2024-09-12T09:34:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31799.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37567.log: /var/log/ceph/ceph-client.admin.45466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41667.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37567.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31799.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41667.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30395.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41667.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36545.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30395.log.gz 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33471.log 2024-09-12T09:34:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33471.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41189.log 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44564.log 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T09:34:41.066 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41189.log.gz 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40914.log 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44564.log: /var/log/ceph/ceph-client.admin.40914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40990.log 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44564.log.gz 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40914.log.gz 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41504.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz -- 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.31441.log 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44568.log 2024-09-12T09:34:41.068 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40990.log: /var/log/ceph/ceph-client.admin.31441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35394.log 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31441.log.gz 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44568.log.gz 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33677.log 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31028.log 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35394.log: /var/log/ceph/ceph-client.admin.33677.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35394.log.gz /var/log/ceph/ceph-client.admin.31306.log 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33677.log.gz 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.31028.log: 95.6% -- replaced with /var/log/ceph/ceph-client.0.31028.log.gz 2024-09-12T09:34:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31936.log 2024-09-12T09:34:41.070 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.4.log: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-12T09:34:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44764.log 2024-09-12T09:34:41.090 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31306.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31936.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.31936.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44764.log: /var/log/ceph/ceph-client.admin.45544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40357.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44764.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32132.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42621.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30214.log 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32132.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40357.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32132.log.gz 2024-09-12T09:34:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42621.log.gz 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30214.log: /var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30214.log.gz 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz /var/log/ceph/ceph-client.admin.34601.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45440.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40642.log: /var/log/ceph/ceph-client.admin.34601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44576.log 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34601.log.gz/var/log/ceph/ceph-client.admin.45440.log: 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45440.log.gz 2024-09-12T09:34:41.092 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31753.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35782.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44576.log: /var/log/ceph/ceph-client.admin.31753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44576.log.gz 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31753.log.gz 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35782.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35782.log.gz -5 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30940.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33951.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34434.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30940.log.gz 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36409.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34434.log: /var/log/ceph/ceph-client.admin.36409.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36409.log.gz -- replaced with /var/log/ceph/ceph-client.admin.34434.log.gz 2024-09-12T09:34:41.093 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31626.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35145.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31626.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26488.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31626.log.gz 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35145.log.gz 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40289.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41857.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.26488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26488.log.gz 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36852.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41857.log.gz 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-09-12T09:34:41.094 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42957.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36852.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45620.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42957.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39815.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45620.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39815.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40890.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44375.log 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2024-09-12T09:34:41.095 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40890.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44375.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26396.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44783.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44998.log: /var/log/ceph/ceph-client.admin.26396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26396.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44783.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42810.log 2024-09-12T09:34:41.096 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34342.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42810.log.gz/var/log/ceph/ceph-client.admin.33795.log: 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42698.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32760.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34342.log: /var/log/ceph/ceph-client.admin.42698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34342.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42698.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42785.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31372.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32760.log: /var/log/ceph/ceph-client.admin.42785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32760.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41295.log 2024-09-12T09:34:41.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40153.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31372.log.gz 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41295.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33151.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40153.log.gz 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42111.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33336.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33151.log: /var/log/ceph/ceph-client.admin.42111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33151.log.gz 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42111.log.gz 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30836.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33336.log.gz 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44374.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.30615.log 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.098 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30836.log.gz 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33023.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30615.log.gz 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37110.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33023.log.gz 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32401.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37110.log.gz 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32401.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.099 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-09-12T09:34:41.109 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-09-12T09:34:41.109 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-09-12T09:34:41.109 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36384.log: /var/log/ceph/ceph-client.admin.37334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-09-12T09:34:41.109 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-09-12T09:34:41.109 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42983.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44369.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42983.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30199.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44369.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40438.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30199.log: /var/log/ceph/ceph-client.admin.41037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30199.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30421.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31832.log 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40438.log: /var/log/ceph/ceph-client.admin.30421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40438.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30421.log.gz 2024-09-12T09:34:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35537.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36743.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31832.log.gz 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35537.log.gz 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21450.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36743.log.gz 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45154.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.21450.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21450.log.gz 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40177.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45154.log.gz 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40085.log 2024-09-12T09:34:41.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30422.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40177.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40085.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32886.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30422.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42308.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32886.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42308.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30209.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30213.log 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-09-12T09:34:41.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30209.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30213.log.gz 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38029.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34292.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.38029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42363.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34292.log.gz 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34779.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42363.log.gz 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-09-12T09:34:41.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34779.log.gz 2024-09-12T09:34:41.116 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31516.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43984.log 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.38029.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31516.log.gz 2024-09-12T09:34:41.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32629.log 2024-09-12T09:34:41.118 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-09-12T09:34:41.118 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43984.log: /var/log/ceph/ceph-client.admin.32629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32629.log.gz 2024-09-12T09:34:41.118 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43984.log.gz 2024-09-12T09:34:41.119 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33524.log 2024-09-12T09:34:41.119 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31684.log 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33524.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36644.log 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31684.log.gz 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36644.log.gz 2024-09-12T09:34:41.120 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30629.log 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41356.log 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-09-12T09:34:41.122 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30629.log.gz 2024-09-12T09:34:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41356.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32273.log 2024-09-12T09:34:41.123 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.123 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42597.log 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30206.log 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32273.log: /var/log/ceph/ceph-client.admin.42597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32273.log.gz 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42597.log.gz 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34651.log 2024-09-12T09:34:41.124 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-09-12T09:34:41.125 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30206.log.gz 2024-09-12T09:34:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34651.log.gz 2024-09-12T09:34:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31838.log 2024-09-12T09:34:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-09-12T09:34:41.126 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44570.log 2024-09-12T09:34:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34184.log 2024-09-12T09:34:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31838.log: /var/log/ceph/ceph-client.admin.44570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31838.log.gz 2024-09-12T09:34:41.127 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44570.log.gz 2024-09-12T09:34:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.37813.log 2024-09-12T09:34:41.127 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34184.log.gz 2024-09-12T09:34:41.128 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30200.log 2024-09-12T09:34:41.128 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40128.log 2024-09-12T09:34:41.129 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.37813.log: /var/log/ceph/ceph-client.admin.30200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30200.log.gz 2024-09-12T09:34:41.130 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/objectstore_tool.37813.log.gz 2024-09-12T09:34:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40709.log 2024-09-12T09:34:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31898.log 2024-09-12T09:34:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40128.log.gz 2024-09-12T09:34:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-09-12T09:34:41.132 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40709.log.gz 2024-09-12T09:34:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31898.log.gz 2024-09-12T09:34:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37202.log 2024-09-12T09:34:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-09-12T09:34:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-09-12T09:34:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37202.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-09-12T09:34:41.133 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.30211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-09-12T09:34:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-09-12T09:34:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31163.log 2024-09-12T09:34:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-09-12T09:34:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-09-12T09:34:41.136 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31163.log.gz 2024-09-12T09:34:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.37593.log 2024-09-12T09:34:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31475.log 2024-09-12T09:34:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-09-12T09:34:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/objectstore_tool.37593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-09-12T09:34:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31475.log.gz 2024-09-12T09:34:41.139 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42836.log: 0.0% 94.8% -- replaced with /var/log/ceph/objectstore_tool.37593.log.gz 2024-09-12T09:34:41.139 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-09-12T09:34:41.195 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-12T09:34:41.312 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-12T09:34:41.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-12T09:34:41.390 INFO:teuthology.orchestra.run.smithi007.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-12T09:34:41.869 INFO:teuthology.orchestra.run.smithi115.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-12T09:34:42.208 INFO:teuthology.orchestra.run.smithi007.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-12T09:34:42.664 INFO:teuthology.orchestra.run.smithi007.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-12T09:34:44.097 INFO:teuthology.orchestra.run.smithi115.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-12T09:34:45.046 INFO:teuthology.orchestra.run.smithi007.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-12T09:34:45.588 INFO:teuthology.orchestra.run.smithi007.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-12T09:34:45.589 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-12T09:34:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m4.663s 2024-09-12T09:34:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m12.041s 2024-09-12T09:34:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.545s 2024-09-12T09:34:45.904 INFO:teuthology.orchestra.run.smithi115.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-12T09:34:47.139 INFO:teuthology.orchestra.run.smithi115.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-12T09:34:47.140 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-12T09:34:47.140 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m6.182s 2024-09-12T09:34:47.141 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m14.821s 2024-09-12T09:34:47.141 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.592s 2024-09-12T09:34:47.141 INFO:tasks.ceph:Archiving logs... 2024-09-12T09:34:47.142 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi007/log 2024-09-12T09:34:47.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T09:34:48.603 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi115/log 2024-09-12T09:34:48.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T09:34:50.138 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T09:34:50.147 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T09:34:50.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T09:34:50.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T09:34:50.241 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T09:34:50.250 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T09:34:50.250 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T09:34:50.253 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:+hv01.front.sepi 67.205.162.81 3 u 33 64 377 0.063 +2.323 0.378 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:*hv02.front.sepi 63.231.80.2 3 u 44 64 377 0.086 -3.095 2.093 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:+hv03.front.sepi 74.6.168.72 3 u 36 64 377 0.062 -3.007 0.492 2024-09-12T09:34:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:============================================================================== 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:+hv01.front.sepi 67.205.162.81 3 u 36 64 377 0.112 +2.376 0.642 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:*hv02.front.sepi 63.231.80.2 3 u 41 64 377 0.085 -2.318 1.084 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout:+hv03.front.sepi 74.6.168.72 3 u 38 64 377 0.081 -2.603 0.698 2024-09-12T09:34:50.267 INFO:teuthology.orchestra.run.smithi115.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T09:34:50.267 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T09:34:50.276 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T09:34:50.277 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T09:34:50.303 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T09:34:50.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T09:34:50.370 INFO:teuthology.task.internal:Duration was 963.217615 seconds 2024-09-12T09:34:50.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T09:34:50.404 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T09:34:50.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T09:34:50.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T09:34:50.443 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T09:34:50.443 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-12T09:34:50.443 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T09:34:50.495 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-12T09:34:50.495 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T09:34:50.506 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T09:34:50.506 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T09:34:50.540 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T09:34:50.579 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T09:34:50.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T09:34:50.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T09:34:50.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T09:34:50.708 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T09:34:50.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T09:34:50.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T09:34:50.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T09:34:50.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T09:34:50.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T09:34:50.773 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-12T09:34:50.784 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-12T09:34:50.804 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T09:34:50.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:34:50.846 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T09:34:50.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T09:34:50.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T09:34:50.859 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T09:34:50.860 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi007 2024-09-12T09:34:50.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T09:34:50.917 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-12_04:46:10-rbd-wip-dis-testing-distro-default-smithi/7902510/remote/smithi115 2024-09-12T09:34:50.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T09:34:50.938 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T09:34:50.938 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T09:34:50.952 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T09:34:50.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T09:34:50.993 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T09:34:50.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T09:34:51.029 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T09:34:51.030 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T09:34:51.032 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T09:34:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 09:34 /home/ubuntu/cephtest 2024-09-12T09:34:51.038 INFO:teuthology.orchestra.run.smithi115.stdout: 658633 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 09:34 /home/ubuntu/cephtest 2024-09-12T09:34:51.040 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T09:34:51.063 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T09:34:51.133 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 963.217615365982 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-12T09:34:51.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T09:34:51.218 INFO:teuthology.run:pass